-- Logs begin at Mon 2022-05-23 14:54:46 UTC, end at Mon 2022-05-23 16:23:08 UTC. -- May 23 15:35:40.978396 dv-cinder-842119-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 23 15:35:40.987373 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 23 15:35:40.990000 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 15:35:40 2022] *** May 23 15:35:40.990157 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 15:35:40.990321 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 15:35:40.990441 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: nodename: dv-cinder-842119-2-iscsi May 23 15:35:40.990546 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: machine: x86_64 May 23 15:35:40.990655 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: clock source: unix May 23 15:35:40.990756 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: pcre jit disabled May 23 15:35:40.990857 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: detected number of CPU cores: 4 May 23 15:35:40.990957 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: current working directory: / May 23 15:35:40.991058 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: detected binary path: /usr/bin/uwsgi-core May 23 15:35:40.991167 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: your processes number limit is 47680 May 23 15:35:40.991285 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: your memory page size is 4096 bytes May 23 15:35:40.991285 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: detected max file descriptor number: 2048 May 23 15:35:40.991285 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: lock engine: pthread robust mutexes May 23 15:35:40.991505 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: thunder lock: enabled May 23 15:35:40.991622 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 23 15:35:40.991822 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 15:35:41.036400 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: Python main interpreter initialized at 0x562ef5ff89c0 May 23 15:35:41.036400 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: python threads support enabled May 23 15:35:41.036400 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: your server socket listen backlog is limited to 100 connections May 23 15:35:41.036400 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: your mercy for graceful operations on workers is 90 seconds May 23 15:35:41.036731 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: mapped 403077 bytes (393 KB) for 2 cores May 23 15:35:41.036805 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: *** Operational MODE: preforking *** May 23 15:35:41.037067 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: *** uWSGI is running in multiple interpreter mode *** May 23 15:35:41.037067 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: spawned uWSGI master process (pid: 87061) May 23 15:35:41.037262 dv-cinder-842119-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 23 15:35:41.040045 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: spawned uWSGI worker 1 (pid: 87073, cores: 1) May 23 15:35:41.040045 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: spawned uWSGI worker 2 (pid: 87074, cores: 1) May 23 15:35:41.040045 dv-cinder-842119-2-iscsi devstack@keystone.service[87061]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 15:35:46.644312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:35:46.644312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: from cryptography.utils import int_from_bytes May 23 15:35:46.645083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:35:46.645083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: from cryptography.utils import int_from_bytes May 23 15:35:46.675704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:35:46.675704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: from cryptography.utils import int_from_bytes May 23 15:35:46.676181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 15:35:46.676181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: from cryptography.utils import int_from_bytes May 23 15:35:46.694642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 15:35:46.694843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 15:35:46.695038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] command line args: [] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 15:35:46.695307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 15:35:46.695484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ================================================================================ {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 15:35:46.695743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] admin_token = **** {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.695987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] config_dir = [] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.696172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.696379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] config_source = [] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.696570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] debug = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.696774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.697264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.697360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.697559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.697806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.698035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] list_limit = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.698213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] log_config_append = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.698390 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.698565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] log_dir = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.698741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] log_file = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.698976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.699221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.699460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.699706 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.699974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.700152 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.700434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.700671 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.700903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.701137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.701368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.701598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.701842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.702085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.702358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.702846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.703029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] publish_errors = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.703260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.703488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.703665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.703911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.704145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.704394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] standard_threads = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.704627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.704869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.705044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.705220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] use_journal = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.705394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] use_json = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.705579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] use_stderr = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.705742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] use_syslog = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.705970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.706222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.706458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.706715 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.706952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.707201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.707439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.707685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.707919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.708163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.708440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.708691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.708925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.709171 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.709406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.709651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.709896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.710131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.710384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.710634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.710878 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.711111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.711361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.711618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.711869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.712118 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.712349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.712600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.712829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.713060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.713307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.713554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.713782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.714011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.714256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.714501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.714735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.714981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.715210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.715980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.716217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.716482 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.716714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.716942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.717173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.717408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.717655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.717885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.718120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.718349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.718584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.718814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.719061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.719289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.719524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.719759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.719992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.720226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.720487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.720718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.720963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.721196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.721430 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.721657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.721911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.722143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.722371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.722605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.722827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 15:35:46.722913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.723024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 15:35:46.723110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] command line args: [] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 15:35:46.723299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.723481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 15:35:46.723563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.723657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ================================================================================ {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 15:35:46.723807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.723908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] admin_token = **** {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.723987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.724151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] config_dir = [] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.724234 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.724337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.724494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.724572 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] config_source = [] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.724702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.724779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] debug = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.724910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.725017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.725146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.725223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.725323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.725400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.725574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.725765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.725846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.725916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.726034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] list_limit = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.726112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.726211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] log_config_append = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.726297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.726389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.726489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.726565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] log_dir = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.726727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] log_file = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.726802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.726961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.727038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.727197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.727277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.727433 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.727510 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.727673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.727750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.727851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.727927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.728031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.728136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.728265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.728462 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.728568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.728677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.728774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.728886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.728986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.729080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.729195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.729269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.729417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.729507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.729642 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.729720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.729886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.729967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.730157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.730248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.730391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.730643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.730724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.730851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] publish_errors = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.730927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.731109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.731717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.731847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.731978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] auth.external = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.732075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.732155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.732265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.732440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.732552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.732687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.732774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] standard_threads = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.732882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] auth.password = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.732961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.733112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] auth.token = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.733193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.733352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.733432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.733556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] use_journal = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.733631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.733752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] use_json = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.733827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.733941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] use_stderr = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.734021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] use_syslog = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.734102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.734221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 15:35:46.734348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.734476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.734554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.734719 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.734805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.734951 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] credential.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.735037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.735196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.735281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.735441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.735525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.735678 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.735755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.735913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.735989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.736145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.736299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.736399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.736482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.736650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.736726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.736896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.737027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.737130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.737206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.737416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.737501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.741596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.741721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.741804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.741878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.741949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742399 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] federation.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.742992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743069 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743218 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.743956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.744787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.745994 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746498 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746857 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.746966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747118 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747510 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.747964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.748098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.748215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.748419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.748500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.748572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.748644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.748723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.748798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.748878 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.748960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.749985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.750908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.751065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.751144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.751298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.751383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.751533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.751612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.751767 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.751853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.752012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.752097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.752243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.752321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.752495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.752576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.752735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.752825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.752973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.753052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.753204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.753281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.753435 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.753520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.753666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.753743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.753897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.753974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.754133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.754219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.754380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.754456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.754619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.754695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.754856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.754940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.755097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.755176 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.755336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.755414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.755568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.755654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.755797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.755875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.756025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.756101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.756256 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.756343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.756512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.756592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.756741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.756817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.756967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.757056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.757208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.757287 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.757441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.757771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.757939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.758025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.758176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.758256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.758405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.758482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.758651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.758738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.758889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.758968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.759118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] auth.external = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.759196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.759348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.759435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.759582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.759658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.759828 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.759906 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.760075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] auth.password = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.760163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.760311 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.760421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] auth.token = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.760575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.760663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.760811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.760891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.761053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.761129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.761281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.761358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.761514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.761592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.761750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.761834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.761980 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.762057 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.762768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] credential.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.762899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.762998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.763117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.763223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.763300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.763463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.763545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] resource.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.763701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.763849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.763927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.764102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.764193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.764342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.764443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.764616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.764695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.764870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.774395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.774513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.774595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.774657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.774715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.774792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.774873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.774946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] role.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] role.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] federation.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775887 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.775957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.776976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777339 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.777950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.778523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.778602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.778668 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.778737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.778807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.778886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.778959 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779325 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779325 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.779962 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780754 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780825 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] token.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.780966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.781955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782364 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.782993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.connection = **** {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783730 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.783944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784236 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784460 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784826 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.784966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.785982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87073) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 15:35:46.786318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.786789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.787065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.787065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.787273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.787520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.787758 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.788109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.788291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.788497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.788738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.788974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.789211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.789524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.789807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.790195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.790512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.790755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] resource.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.791009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.791258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.791496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.791746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.791996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.792231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.792501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.792741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.792993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.793226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] role.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.793533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] role.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.793727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.793977 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.794226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.794473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.794725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.794971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.795219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.795464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.795711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.795942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.796189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.796466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.796717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.796964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.797367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.797604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.797854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.798103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.798351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.798599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.798844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.799098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.799350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.799589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.799820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.800053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.800283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.800564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.800821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.801069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.801306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.801541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.801770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.802004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] token.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.802240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.802488 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.802723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.802981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.803228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.803465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.803709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.803950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.804195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.804455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.804700 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.804934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.805178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.805426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.805656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.805898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.806186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.806448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.connection = **** {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.806695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.806944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.807184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.807427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.807665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.807903 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.808142 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.808396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.808646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.808883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.809163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.809398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.809647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.809884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.810128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.810358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.810597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.810829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 15:35:46.810998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87074) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 15:35:46.814039 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.814273 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.822964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.836006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.836238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.836490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.846911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.851933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.852168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.865288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.865523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.905394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.908874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.956794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.957026 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.957219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:46.957456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.023521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.023715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.023805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.023949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.024033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.024161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.024262 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.024392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.024479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.024610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.024694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.024824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.048317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.048495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.048621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.048703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.048799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.048938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.049021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.049096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.049187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.049275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.049407 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.049533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.049636 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.049720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.060878 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.060998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.073288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.073519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.073701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.073787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.073935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.074020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.074147 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.074229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.074361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.074453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.074573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87073) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.074656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.075326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.075547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87074) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 15:35:47.093255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.093462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:35:47.093801 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.093933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.094066 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:35:47.094200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:35:47.094342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.094475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.094606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.094736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 15:35:47.094867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.094997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.095136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.095267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.095408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.095597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.106627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.106899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.107167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.107448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.107687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.107964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.108231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.108539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.115512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.116570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.118789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.119066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.119349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.119629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.119804 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.119910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.120080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.120170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.120380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.120555 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.120656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.120828 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.120921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.121098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.121292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.121372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.121563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.121700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.121883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.122155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.122461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.122728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.148540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.149606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.150268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.150546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.151308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.151585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.156388 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.156626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.156908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.157381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.157658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.157938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.164116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.164415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.165187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.165464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.168896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.169297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.169580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.169868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.169949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.170161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.170324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.170403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.170581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.170664 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.170860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.170942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.171132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.171401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.171666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.171930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.189889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.190162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.190907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.191180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.194825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.195800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.198630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.198929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.199438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.199742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.204642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.205345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.208201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.208977 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.211957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.212263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.212573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.212712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.212859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.213009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.213143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.213300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.213420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.213583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.213863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.214146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.232221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.232525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.232792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.232943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.233055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.233225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.233489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.233749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.241001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.241705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.242693 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.242975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.243391 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.243664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.246028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.246309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.246588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.247224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.247372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.247505 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.247654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.247784 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.248061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.248331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.261240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.261519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.261803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.261887 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.262079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.262163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.262924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.263020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.263204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.263290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.263485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.263576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.263775 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.264055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.281399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.281931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.284672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.285186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.287955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.288471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.289566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.290090 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.291208 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.291495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.291689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.291780 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.291974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.292271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.299139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.299669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.300771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.301304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.303983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.304278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.304525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.304610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.304813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.304914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.305093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.305407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.314703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.314989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.315372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.315626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.315914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.316298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.323193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87073) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.323471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.323750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.324025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.324106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87074) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 15:35:47.324294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.324396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.324659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.324778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.325484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.325569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.325756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.325836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.326024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.326114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.326288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.326368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.326554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.326643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.326821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.326901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.327090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.327169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.327354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87073) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.327437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.327611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.327875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.328152 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87074) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 15:35:47.365715 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562ef5ff89c0 pid: 87073 (default app) May 23 15:35:47.366489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562ef5ff89c0 pid: 87074 (default app) May 23 15:35:47.368941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c704d5f-b689-4164-b073-689db3fcb6a1 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:47.369236 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c704d5f-b689-4164-b073-689db3fcb6a1 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:47.369515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c704d5f-b689-4164-b073-689db3fcb6a1 None None] PATH_INFO: `/v3` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:47.374313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Mon May 23 15:35:47 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:35:56.055816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9862d3-18b3-4a76-a49b-5a7c21fbd62e None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.056406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9862d3-18b3-4a76-a49b-5a7c21fbd62e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.056406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9862d3-18b3-4a76-a49b-5a7c21fbd62e None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.061353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:35:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:56.066926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.067364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.067506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.135017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: INFO dbcounter [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] Registered counter for database keystone May 23 15:35:56.156503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG oslo_db.sqlalchemy.engines [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87073) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:35:56.162745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writer thread running {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:35:56.338571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG passlib.handlers.bcrypt [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87073) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 15:35:56.338934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG passlib.handlers.bcrypt [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87073) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 15:35:56.400213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:56.401850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:56.524651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:35:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 23 15:35:56.687939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9862d3-18b3-4a76-a49b-5a7c21fbd62e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.755279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: INFO dbcounter [None req-aa9862d3-18b3-4a76-a49b-5a7c21fbd62e None None] Registered counter for database keystone May 23 15:35:56.774798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG oslo_db.sqlalchemy.engines [None req-aa9862d3-18b3-4a76-a49b-5a7c21fbd62e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87074) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 15:35:56.781567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writer thread running {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 15:35:56.900981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9862d3-18b3-4a76-a49b-5a7c21fbd62e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:56.905839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4eba71-d8b7-43d3-8375-798851f1d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:56.906499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eba71-d8b7-43d3-8375-798851f1d416 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:56.906781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eba71-d8b7-43d3-8375-798851f1d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:56.907055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eba71-d8b7-43d3-8375-798851f1d416 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:56.915029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4eba71-d8b7-43d3-8375-798851f1d416 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:56.965974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4a4eba71-d8b7-43d3-8375-798851f1d416 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 15:35:56.966258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4eba71-d8b7-43d3-8375-798851f1d416 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:56.975485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Mon May 23 15:35:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:35:56.981464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:56.995992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedd1bb-3e8e-452d-abb1-a6fa7c293f17 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:57.000957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a7188-79eb-427c-b041-c85963a1cd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:57.002128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7188-79eb-427c-b041-c85963a1cd8f admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:57.002427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7188-79eb-427c-b041-c85963a1cd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:57.002704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a7188-79eb-427c-b041-c85963a1cd8f admin admin] PATH_INFO: `/v3/projects/6e65bbb7bace4fb6a7f71d68e850edbd` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:57.004215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a7188-79eb-427c-b041-c85963a1cd8f admin admin] RBAC: Authorizing `identity:get_project(project_id=6e65bbb7bace4fb6a7f71d68e850edbd)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:57.161832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WARNING keystone.common.rbac_enforcer.enforcer [None req-603a7188-79eb-427c-b041-c85963a1cd8f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 15:35:57.162479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a7188-79eb-427c-b041-c85963a1cd8f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:57.166875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Mon May 23 15:35:56 2022] GET /identity/v3/projects/6e65bbb7bace4fb6a7f71d68e850edbd => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:35:59.304175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cbea8-277b-49a2-ac50-a97b46e4e953 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.304578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cbea8-277b-49a2-ac50-a97b46e4e953 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.304767 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cbea8-277b-49a2-ac50-a97b46e4e953 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.308826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:35:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:35:59.314549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f18c48-8b1b-4338-9d17-4b6baa9655b4 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.314991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f18c48-8b1b-4338-9d17-4b6baa9655b4 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.315121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f18c48-8b1b-4338-9d17-4b6baa9655b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.333601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e0f18c48-8b1b-4338-9d17-4b6baa9655b4 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:35:59.335114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e0f18c48-8b1b-4338-9d17-4b6baa9655b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:35:59.343432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:35:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 15:35:59.365199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cbea8-277b-49a2-ac50-a97b46e4e953 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.376940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cbea8-277b-49a2-ac50-a97b46e4e953 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.381108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd5af2-94dc-4ed2-8154-c599f6fd0c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.381729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd5af2-94dc-4ed2-8154-c599f6fd0c1a admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.382014 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd5af2-94dc-4ed2-8154-c599f6fd0c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.382292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd5af2-94dc-4ed2-8154-c599f6fd0c1a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.390123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd5af2-94dc-4ed2-8154-c599f6fd0c1a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:59.391860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd5af2-94dc-4ed2-8154-c599f6fd0c1a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:59.400409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Mon May 23 15:35:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:35:59.405938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f18c48-8b1b-4338-9d17-4b6baa9655b4 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:35:59.414836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f18c48-8b1b-4338-9d17-4b6baa9655b4 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:35:59.419805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9735cd6-b5c4-4459-bfe2-d94c433ca555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:35:59.420699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9735cd6-b5c4-4459-bfe2-d94c433ca555 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:35:59.420986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9735cd6-b5c4-4459-bfe2-d94c433ca555 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:35:59.421260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9735cd6-b5c4-4459-bfe2-d94c433ca555 admin admin] PATH_INFO: `/v3/users/0840c2e542b841edbc74432280137f72` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:35:59.422647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9735cd6-b5c4-4459-bfe2-d94c433ca555 admin admin] RBAC: Authorizing `identity:get_user(user_id=0840c2e542b841edbc74432280137f72)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:35:59.424500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9735cd6-b5c4-4459-bfe2-d94c433ca555 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:35:59.426038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Mon May 23 15:35:59 2022] GET /identity/v3/users/0840c2e542b841edbc74432280137f72 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:01.663972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe51242-c584-45eb-974d-a541e6ee7fd2 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:01.664475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe51242-c584-45eb-974d-a541e6ee7fd2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:01.664475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe51242-c584-45eb-974d-a541e6ee7fd2 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:01.665817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:01.671571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f4f1b-64b3-437d-bfb1-028cb925bf08 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:01.671999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f4f1b-64b3-437d-bfb1-028cb925bf08 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:01.672263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f4f1b-64b3-437d-bfb1-028cb925bf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:01.676135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87bde4-b4b9-47ad-b7d0-5d6e2527260f None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:01.676448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87bde4-b4b9-47ad-b7d0-5d6e2527260f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:01.676731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87bde4-b4b9-47ad-b7d0-5d6e2527260f None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:01.677987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:01.683004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecfaa0-c5b1-49a4-80c0-a115679ad74f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:01.683296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecfaa0-c5b1-49a4-80c0-a115679ad74f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:01.683578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecfaa0-c5b1-49a4-80c0-a115679ad74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:01.688979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d29f4f1b-64b3-437d-bfb1-028cb925bf08 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:01.690166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d29f4f1b-64b3-437d-bfb1-028cb925bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:01.722612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG passlib.handlers.bcrypt [None req-25ecfaa0-c5b1-49a4-80c0-a115679ad74f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87074) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 15:36:01.723039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG passlib.handlers.bcrypt [None req-25ecfaa0-c5b1-49a4-80c0-a115679ad74f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87074) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 15:36:01.769612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:01.770216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-25ecfaa0-c5b1-49a4-80c0-a115679ad74f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:01.771819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-25ecfaa0-c5b1-49a4-80c0-a115679ad74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:01.792736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f4f1b-64b3-437d-bfb1-028cb925bf08 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:01.816057 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:01.870356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:01.892656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ecfaa0-c5b1-49a4-80c0-a115679ad74f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:01.896378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f4f1b-64b3-437d-bfb1-028cb925bf08 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:01.930303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f4f34-eac0-4872-a5de-b24d80c09df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:01.931037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f4f34-eac0-4872-a5de-b24d80c09df7 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:01.931327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f4f34-eac0-4872-a5de-b24d80c09df7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:01.931609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f4f34-eac0-4872-a5de-b24d80c09df7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:01.942629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f4f34-eac0-4872-a5de-b24d80c09df7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-446f4f34-eac0-4872-a5de-b24d80c09df7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:01.954282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:01.955197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:01.956010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:36:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:01.961267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f4f34-eac0-4872-a5de-b24d80c09df7 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:01.993560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ecfaa0-c5b1-49a4-80c0-a115679ad74f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:01.996414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:02.031174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42fe30-5268-4002-8a95-229b511a549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:02.032111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42fe30-5268-4002-8a95-229b511a549f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:02.032429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42fe30-5268-4002-8a95-229b511a549f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:02.032716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42fe30-5268-4002-8a95-229b511a549f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:02.041818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:02.060848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f4f34-eac0-4872-a5de-b24d80c09df7 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:02.093383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba035bc7-fa2c-40fb-8de1-7b1b96e62c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:02.094041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba035bc7-fa2c-40fb-8de1-7b1b96e62c34 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:02.094327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba035bc7-fa2c-40fb-8de1-7b1b96e62c34 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:02.094607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba035bc7-fa2c-40fb-8de1-7b1b96e62c34 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:02.095347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba035bc7-fa2c-40fb-8de1-7b1b96e62c34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:02.095654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba035bc7-fa2c-40fb-8de1-7b1b96e62c34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:02.097067 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba035bc7-fa2c-40fb-8de1-7b1b96e62c34 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:02.101492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42fe30-5268-4002-8a95-229b511a549f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:02.102983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42fe30-5268-4002-8a95-229b511a549f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:02.108964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:02.114361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba035bc7-fa2c-40fb-8de1-7b1b96e62c34 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:02.201162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 8/16] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:36:02.206459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42fe30-5268-4002-8a95-229b511a549f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:02.209053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:02.212149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba035bc7-fa2c-40fb-8de1-7b1b96e62c34 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:02.246516 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471740b2-4f11-4581-89e1-5f6e9a4fdd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:02.247414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471740b2-4f11-4581-89e1-5f6e9a4fdd7e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:02.247701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471740b2-4f11-4581-89e1-5f6e9a4fdd7e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:02.247979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471740b2-4f11-4581-89e1-5f6e9a4fdd7e None admin] PATH_INFO: `/v3/users/0840c2e542b841edbc74432280137f72` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:02.249220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471740b2-4f11-4581-89e1-5f6e9a4fdd7e None admin] RBAC: Authorizing `identity:get_user(user_id=0840c2e542b841edbc74432280137f72)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:02.250678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471740b2-4f11-4581-89e1-5f6e9a4fdd7e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:02.252284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:02 2022] GET /identity/v3/users/0840c2e542b841edbc74432280137f72 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:02.257414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471740b2-4f11-4581-89e1-5f6e9a4fdd7e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:02.272504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:02.306705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42fe30-5268-4002-8a95-229b511a549f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:02.339791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1156870-b222-49d8-8834-ecaffcf6da80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:02.341145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1156870-b222-49d8-8834-ecaffcf6da80 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:02.341440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1156870-b222-49d8-8834-ecaffcf6da80 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:02.341721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1156870-b222-49d8-8834-ecaffcf6da80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:02.358932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471740b2-4f11-4581-89e1-5f6e9a4fdd7e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:02.360299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1156870-b222-49d8-8834-ecaffcf6da80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:02.361790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1156870-b222-49d8-8834-ecaffcf6da80 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-e1156870-b222-49d8-8834-ecaffcf6da80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:36:02.381989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:02.383082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:02.383966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:36:02.383966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:36:02.383966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:36:02.383966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:02.383966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:02.389549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1156870-b222-49d8-8834-ecaffcf6da80 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:02.392389 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde14422-5bf1-419e-bc0a-d42e0a77a125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:02.393193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde14422-5bf1-419e-bc0a-d42e0a77a125 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:02.393486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde14422-5bf1-419e-bc0a-d42e0a77a125 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:02.393770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde14422-5bf1-419e-bc0a-d42e0a77a125 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:02.394973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde14422-5bf1-419e-bc0a-d42e0a77a125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:02.396454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde14422-5bf1-419e-bc0a-d42e0a77a125 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:02.398041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:02.404028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde14422-5bf1-419e-bc0a-d42e0a77a125 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:02.449676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:02.488495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:02.493764 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1156870-b222-49d8-8834-ecaffcf6da80 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:02.507060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde14422-5bf1-419e-bc0a-d42e0a77a125 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:02.527856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78058dc-bd33-4066-972f-52b85c04bd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:02.528558 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78058dc-bd33-4066-972f-52b85c04bd5c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:02.528844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78058dc-bd33-4066-972f-52b85c04bd5c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:02.529119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78058dc-bd33-4066-972f-52b85c04bd5c None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:02.529842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78058dc-bd33-4066-972f-52b85c04bd5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:02.530146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78058dc-bd33-4066-972f-52b85c04bd5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:02.531525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78058dc-bd33-4066-972f-52b85c04bd5c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:02.540494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1757f5-0e1e-4201-8989-6d649d6314c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:02.541153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1757f5-0e1e-4201-8989-6d649d6314c8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:02.541446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1757f5-0e1e-4201-8989-6d649d6314c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:02.541729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1757f5-0e1e-4201-8989-6d649d6314c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:02.542565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1757f5-0e1e-4201-8989-6d649d6314c8 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.domain.id=default, user.id=0840c2e542b841edbc74432280137f72) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:02.542881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1757f5-0e1e-4201-8989-6d649d6314c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:02.544907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1757f5-0e1e-4201-8989-6d649d6314c8 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:02.550220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:02.551510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 11/21] 10.222.0.23 () {56 vars in 1322 bytes} [Mon May 23 15:36:02 2022] GET /identity/v3/role_assignments?user.id=0840c2e542b841edbc74432280137f72&scope.domain.id=default&role.id=04c11dea81274a69974ad5c639f2c60b => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:04.941197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcdd6d-31b2-45d1-ae20-c6f737604ac7 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:04.942210 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcdd6d-31b2-45d1-ae20-c6f737604ac7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:04.942716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcdd6d-31b2-45d1-ae20-c6f737604ac7 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:04.944282 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:04.949682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53069280-d970-4533-bdeb-7389e2e5bc59 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:04.950553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53069280-d970-4533-bdeb-7389e2e5bc59 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:04.951046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53069280-d970-4533-bdeb-7389e2e5bc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:04.976139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-53069280-d970-4533-bdeb-7389e2e5bc59 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:04.977715 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-53069280-d970-4533-bdeb-7389e2e5bc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:05.062638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 12/23] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:05.095939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcdd6d-31b2-45d1-ae20-c6f737604ac7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:05.215012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37caf3-8790-4300-9591-f38ac0762b7c None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:05.215627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37caf3-8790-4300-9591-f38ac0762b7c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:05.216657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37caf3-8790-4300-9591-f38ac0762b7c None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:05.218221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 13/24] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:05.223506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7065f6d-d5ab-4733-ac85-dfac59e06073 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:05.224038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7065f6d-d5ab-4733-ac85-dfac59e06073 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:05.224564 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7065f6d-d5ab-4733-ac85-dfac59e06073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:05.256379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b7065f6d-d5ab-4733-ac85-dfac59e06073 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:05.257585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b7065f6d-d5ab-4733-ac85-dfac59e06073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:05.387301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 14/25] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:05.394068 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcdd6d-31b2-45d1-ae20-c6f737604ac7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:05.404448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a241a4-dd3e-4fa1-9771-c74bc0969c6a None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:05.404748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a241a4-dd3e-4fa1-9771-c74bc0969c6a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:05.405034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a241a4-dd3e-4fa1-9771-c74bc0969c6a None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:05.406399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 15/26] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:05.417441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39484582-62ec-4ba3-ab90-83a46f1bce97 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:05.417731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39484582-62ec-4ba3-ab90-83a46f1bce97 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:05.418016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39484582-62ec-4ba3-ab90-83a46f1bce97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:05.426851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:05.443942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:05.444519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-39484582-62ec-4ba3-ab90-83a46f1bce97 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:05.449312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-39484582-62ec-4ba3-ab90-83a46f1bce97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:05.472575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a7642-b33b-4080-a9c7-ac428ac0899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:05.474315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a7642-b33b-4080-a9c7-ac428ac0899b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:05.474599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a7642-b33b-4080-a9c7-ac428ac0899b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:05.474868 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a7642-b33b-4080-a9c7-ac428ac0899b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:05.494285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a7642-b33b-4080-a9c7-ac428ac0899b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-0d0a7642-b33b-4080-a9c7-ac428ac0899b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:05.504172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:05.505057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:05.505792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:05.506356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 12/27] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:36:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:05.506931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a7642-b33b-4080-a9c7-ac428ac0899b None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:05.597097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 16/28] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:05.599479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39484582-62ec-4ba3-ab90-83a46f1bce97 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:05.696711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a7642-b33b-4080-a9c7-ac428ac0899b None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:05.727134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39484582-62ec-4ba3-ab90-83a46f1bce97 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:05.739454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a141b1b-f0f9-4b0e-b133-10f140b10472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:05.740256 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a141b1b-f0f9-4b0e-b133-10f140b10472 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:05.740568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a141b1b-f0f9-4b0e-b133-10f140b10472 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:05.740843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a141b1b-f0f9-4b0e-b133-10f140b10472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:05.752185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a141b1b-f0f9-4b0e-b133-10f140b10472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:05.753683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a141b1b-f0f9-4b0e-b133-10f140b10472 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-5a141b1b-f0f9-4b0e-b133-10f140b10472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:36:05.774900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:05.776891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:05.778892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:36:05.778892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:36:05.778892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:36:05.778892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:05.781986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 13/29] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:05.784384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a141b1b-f0f9-4b0e-b133-10f140b10472 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:05.789721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682774-a7f9-4085-9114-a9097479baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:05.790401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682774-a7f9-4085-9114-a9097479baab None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:05.790685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682774-a7f9-4085-9114-a9097479baab None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:05.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682774-a7f9-4085-9114-a9097479baab None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:05.791724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d682774-a7f9-4085-9114-a9097479baab None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:05.792033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d682774-a7f9-4085-9114-a9097479baab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:05.798068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d682774-a7f9-4085-9114-a9097479baab None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:05.801947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:05.814338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 17/30] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:05.818108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682774-a7f9-4085-9114-a9097479baab None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:05.845184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:05.984295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a141b1b-f0f9-4b0e-b133-10f140b10472 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:06.025027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d6a20-2353-4175-b1f4-7fa2fb0a5345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:06.025976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d6a20-2353-4175-b1f4-7fa2fb0a5345 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.026415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d6a20-2353-4175-b1f4-7fa2fb0a5345 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.026751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682774-a7f9-4085-9114-a9097479baab None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:06.027335 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d6a20-2353-4175-b1f4-7fa2fb0a5345 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.028956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d6a20-2353-4175-b1f4-7fa2fb0a5345 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:06.030603 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d6a20-2353-4175-b1f4-7fa2fb0a5345 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:06.071964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f947a9c-a19c-4110-80ce-36ab9d38b369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:06.073078 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f947a9c-a19c-4110-80ce-36ab9d38b369 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.073556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f947a9c-a19c-4110-80ce-36ab9d38b369 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.074011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f947a9c-a19c-4110-80ce-36ab9d38b369 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.074927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f947a9c-a19c-4110-80ce-36ab9d38b369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:06.075427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f947a9c-a19c-4110-80ce-36ab9d38b369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:06.076985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f947a9c-a19c-4110-80ce-36ab9d38b369 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:06.096621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 18/31] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:06.097898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f947a9c-a19c-4110-80ce-36ab9d38b369 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:06.180262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:06.248283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 14/32] 10.222.0.23 () {60 vars in 1131 bytes} [Mon May 23 15:36:05 2022] POST /identity/v3/roles => generated 219 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:06.249452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d6a20-2353-4175-b1f4-7fa2fb0a5345 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:06.274562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f947a9c-a19c-4110-80ce-36ab9d38b369 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:06.362487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea5955-070e-4f17-86c0-b7b0d50820ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:06.363414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea5955-070e-4f17-86c0-b7b0d50820ca None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.363707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea5955-070e-4f17-86c0-b7b0d50820ca None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.363989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea5955-070e-4f17-86c0-b7b0d50820ca None admin] PATH_INFO: `/v3/users/0840c2e542b841edbc74432280137f72` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.367380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea5955-070e-4f17-86c0-b7b0d50820ca None admin] RBAC: Authorizing `identity:get_user(user_id=0840c2e542b841edbc74432280137f72)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:06.368962 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea5955-070e-4f17-86c0-b7b0d50820ca None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:06.371089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 19/33] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:06 2022] GET /identity/v3/users/0840c2e542b841edbc74432280137f72 => generated 247 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:06.372738 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5ecef-0098-4c44-a26e-17cf36a401fc None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.373043 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5ecef-0098-4c44-a26e-17cf36a401fc None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.373327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5ecef-0098-4c44-a26e-17cf36a401fc None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.374831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 20/34] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:06.381194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b77129-76f6-433a-aeef-607db6746c95 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.381485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b77129-76f6-433a-aeef-607db6746c95 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.381769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b77129-76f6-433a-aeef-607db6746c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.416168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-22b77129-76f6-433a-aeef-607db6746c95 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:06.417526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-22b77129-76f6-433a-aeef-607db6746c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:06.435992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:06.466432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d6a20-2353-4175-b1f4-7fa2fb0a5345 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:06.516526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 21/35] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:06.518194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b77129-76f6-433a-aeef-607db6746c95 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:06.559405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5036e4-b1ca-4f2a-8999-5f588d971863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:06.560153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5036e4-b1ca-4f2a-8999-5f588d971863 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.560481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5036e4-b1ca-4f2a-8999-5f588d971863 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.560768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5036e4-b1ca-4f2a-8999-5f588d971863 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.561751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5036e4-b1ca-4f2a-8999-5f588d971863 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:06.563334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5036e4-b1ca-4f2a-8999-5f588d971863 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:06.589797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:06.636763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b77129-76f6-433a-aeef-607db6746c95 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:06.643734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 15/36] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:36:06 2022] POST /identity/v3/projects => generated 301 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:06.649546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5036e4-b1ca-4f2a-8999-5f588d971863 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:06.678801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fe522-fa00-4ca6-b1fc-b438be61b6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:06.679963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fe522-fa00-4ca6-b1fc-b438be61b6dc None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.680492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fe522-fa00-4ca6-b1fc-b438be61b6dc None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.680980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fe522-fa00-4ca6-b1fc-b438be61b6dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.682653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2fe522-fa00-4ca6-b1fc-b438be61b6dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:06.684279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2fe522-fa00-4ca6-b1fc-b438be61b6dc None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:06.686912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 22/37] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:06.688266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7370e-6a59-46d9-9612-a5b095faeba7 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.688586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7370e-6a59-46d9-9612-a5b095faeba7 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.688872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7370e-6a59-46d9-9612-a5b095faeba7 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.690378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 23/38] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:06.691848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1f67-03b4-4042-995e-e019c8077f4f None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.692135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1f67-03b4-4042-995e-e019c8077f4f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.692546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1f67-03b4-4042-995e-e019c8077f4f None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.694053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 24/39] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:06.694804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c1f67-03b4-4042-995e-e019c8077f4f None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:06.777662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5036e4-b1ca-4f2a-8999-5f588d971863 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:06.812085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c1f67-03b4-4042-995e-e019c8077f4f None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:06.816856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460d7e4-745e-444e-ac8d-f916832d146f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:06.817561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d7e4-745e-444e-ac8d-f916832d146f None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.817851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d7e4-745e-444e-ac8d-f916832d146f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.818135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d7e4-745e-444e-ac8d-f916832d146f None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.818911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460d7e4-745e-444e-ac8d-f916832d146f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:06.820331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460d7e4-745e-444e-ac8d-f916832d146f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:06.838601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:06.853083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f68de-9afe-4206-8245-b9029142268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:06.853979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f68de-9afe-4206-8245-b9029142268f None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.854250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f68de-9afe-4206-8245-b9029142268f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.854530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f68de-9afe-4206-8245-b9029142268f None admin] PATH_INFO: `/v3/domains/default/users/0840c2e542b841edbc74432280137f72/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.856775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f68de-9afe-4206-8245-b9029142268f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0840c2e542b841edbc74432280137f72, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:06.858242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f68de-9afe-4206-8245-b9029142268f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:06.905493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:06.910511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 16/40] 10.222.0.23 () {60 vars in 1131 bytes} [Mon May 23 15:36:06 2022] POST /identity/v3/roles => generated 225 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:06.911841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873b9ec-6038-4929-92b9-8b2fa9a1d88d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:06.912140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873b9ec-6038-4929-92b9-8b2fa9a1d88d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:06.912459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873b9ec-6038-4929-92b9-8b2fa9a1d88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:06.930475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-d873b9ec-6038-4929-92b9-8b2fa9a1d88d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:06.932089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d873b9ec-6038-4929-92b9-8b2fa9a1d88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:07.005101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 17/41] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:07.006546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c532ca9-20da-474c-a756-89b26a21a6ad None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:07.006857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c532ca9-20da-474c-a756-89b26a21a6ad None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:07.007179 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c532ca9-20da-474c-a756-89b26a21a6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:07.031273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-4c532ca9-20da-474c-a756-89b26a21a6ad None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:07.032503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-4c532ca9-20da-474c-a756-89b26a21a6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:07.048543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 25/42] 10.222.0.23 () {58 vars in 1361 bytes} [Mon May 23 15:36:06 2022] PUT /identity/v3/domains/default/users/0840c2e542b841edbc74432280137f72/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:07.050781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f68de-9afe-4206-8245-b9029142268f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:07.127811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 18/43] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:07.153516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c532ca9-20da-474c-a756-89b26a21a6ad None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:07.193329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f68de-9afe-4206-8245-b9029142268f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:07.222068 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:07.236558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bfb8c-5fa8-4b54-b726-b1fa6f6cca77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:07.237643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bfb8c-5fa8-4b54-b726-b1fa6f6cca77 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:07.238061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bfb8c-5fa8-4b54-b726-b1fa6f6cca77 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:07.238468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bfb8c-5fa8-4b54-b726-b1fa6f6cca77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:07.240012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1bfb8c-5fa8-4b54-b726-b1fa6f6cca77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:07.241746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1bfb8c-5fa8-4b54-b726-b1fa6f6cca77 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:07.246314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 26/44] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:07.252844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bfb8c-5fa8-4b54-b726-b1fa6f6cca77 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:07.274514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c532ca9-20da-474c-a756-89b26a21a6ad None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:07.313499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85251f2-ef1c-425f-8a07-58381d4d59bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:07.313733 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:07.314185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85251f2-ef1c-425f-8a07-58381d4d59bd None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:07.314479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85251f2-ef1c-425f-8a07-58381d4d59bd None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:07.314898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85251f2-ef1c-425f-8a07-58381d4d59bd None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:07.315528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85251f2-ef1c-425f-8a07-58381d4d59bd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:07.316957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85251f2-ef1c-425f-8a07-58381d4d59bd None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:07.363843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bfb8c-5fa8-4b54-b726-b1fa6f6cca77 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:07.399022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb01795-5bee-4c83-beb8-64c3a058b673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:07.399703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb01795-5bee-4c83-beb8-64c3a058b673 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:07.399990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb01795-5bee-4c83-beb8-64c3a058b673 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:07.400273 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb01795-5bee-4c83-beb8-64c3a058b673 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:07.401028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb01795-5bee-4c83-beb8-64c3a058b673 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:07.402502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb01795-5bee-4c83-beb8-64c3a058b673 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:07.413658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 19/45] 10.222.0.23 () {60 vars in 1131 bytes} [Mon May 23 15:36:07 2022] POST /identity/v3/roles => generated 223 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:07.504387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 27/46] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:36:07 2022] POST /identity/v3/projects => generated 312 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:09.238842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad78b1-2bc0-477e-b321-090d85c54f0f None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:09.239358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad78b1-2bc0-477e-b321-090d85c54f0f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:09.239440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad78b1-2bc0-477e-b321-090d85c54f0f None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:09.240794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 20/47] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:09.246030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db177ac-566f-4213-b35a-264c5be8e6c4 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:09.246447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db177ac-566f-4213-b35a-264c5be8e6c4 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:09.246737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db177ac-566f-4213-b35a-264c5be8e6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:09.264460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-4db177ac-566f-4213-b35a-264c5be8e6c4 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:09.265974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4db177ac-566f-4213-b35a-264c5be8e6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:09.334148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 28/48] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:09.357843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad78b1-2bc0-477e-b321-090d85c54f0f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:09.456901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad78b1-2bc0-477e-b321-090d85c54f0f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:09.476245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:09.490585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6f026-de0d-47c8-933a-f3e9237fc417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:09.490585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6f026-de0d-47c8-933a-f3e9237fc417 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:09.490994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6f026-de0d-47c8-933a-f3e9237fc417 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:09.490994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6f026-de0d-47c8-933a-f3e9237fc417 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:09.503070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6f026-de0d-47c8-933a-f3e9237fc417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-a2c6f026-de0d-47c8-933a-f3e9237fc417 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:09.511622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:09.512567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:09.513852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:09.514431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 21/49] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:36:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:09.520315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db177ac-566f-4213-b35a-264c5be8e6c4 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:09.552069 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:09.619010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db177ac-566f-4213-b35a-264c5be8e6c4 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:09.651546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511ee86-8cca-465c-a45d-5f1cd48dc315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:09.652215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511ee86-8cca-465c-a45d-5f1cd48dc315 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:09.652534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511ee86-8cca-465c-a45d-5f1cd48dc315 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:09.652814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511ee86-8cca-465c-a45d-5f1cd48dc315 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:09.654111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511ee86-8cca-465c-a45d-5f1cd48dc315 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:09.654419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511ee86-8cca-465c-a45d-5f1cd48dc315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:09.655816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511ee86-8cca-465c-a45d-5f1cd48dc315 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:09.663889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 29/50] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:09.669386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6f026-de0d-47c8-933a-f3e9237fc417 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:09.690629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a4f3-086f-479e-ba16-127d27b354e8 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:09.690991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a4f3-086f-479e-ba16-127d27b354e8 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:09.691215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a4f3-086f-479e-ba16-127d27b354e8 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:09.692599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 30/51] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:09.697368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc026110-b1db-4b42-be92-a242538e5a1b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:09.697658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc026110-b1db-4b42-be92-a242538e5a1b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:09.697940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc026110-b1db-4b42-be92-a242538e5a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:09.715115 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-bc026110-b1db-4b42-be92-a242538e5a1b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:09.716347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bc026110-b1db-4b42-be92-a242538e5a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:09.769940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6f026-de0d-47c8-933a-f3e9237fc417 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:09.780995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 31/52] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:09.803067 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc026110-b1db-4b42-be92-a242538e5a1b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:09.803217 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81259189-0f78-4cd6-ad05-87719c07d88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:09.804054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81259189-0f78-4cd6-ad05-87719c07d88b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:09.804337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81259189-0f78-4cd6-ad05-87719c07d88b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:09.804643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81259189-0f78-4cd6-ad05-87719c07d88b None admin] PATH_INFO: `/v3/users/0840c2e542b841edbc74432280137f72` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:09.806055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81259189-0f78-4cd6-ad05-87719c07d88b None admin] RBAC: Authorizing `identity:get_user(user_id=0840c2e542b841edbc74432280137f72)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:09.807497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81259189-0f78-4cd6-ad05-87719c07d88b None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:09.809147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 22/53] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:09 2022] GET /identity/v3/users/0840c2e542b841edbc74432280137f72 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:09.814777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81259189-0f78-4cd6-ad05-87719c07d88b None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:09.830089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:09.863705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:09.903808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc026110-b1db-4b42-be92-a242538e5a1b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:09.915246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81259189-0f78-4cd6-ad05-87719c07d88b None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:09.938632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a001c26-2d30-45f6-9cc1-8f113165762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:09.939428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a001c26-2d30-45f6-9cc1-8f113165762f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:09.939714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a001c26-2d30-45f6-9cc1-8f113165762f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:09.939999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a001c26-2d30-45f6-9cc1-8f113165762f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:09.941241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a001c26-2d30-45f6-9cc1-8f113165762f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:09.942694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a001c26-2d30-45f6-9cc1-8f113165762f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:09.944294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 32/54] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:09.948321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffee46-3c90-40cd-9a5c-9b8341a25bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:09.949171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffee46-3c90-40cd-9a5c-9b8341a25bed None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:09.949462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffee46-3c90-40cd-9a5c-9b8341a25bed None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:09.949620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a001c26-2d30-45f6-9cc1-8f113165762f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:09.949744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffee46-3c90-40cd-9a5c-9b8341a25bed None admin] PATH_INFO: `/v3/domains/default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:09.950994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffee46-3c90-40cd-9a5c-9b8341a25bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:09.952473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffee46-3c90-40cd-9a5c-9b8341a25bed None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:09.954075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 23/55] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:09.959393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffee46-3c90-40cd-9a5c-9b8341a25bed None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:10.039698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:10.048866 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a001c26-2d30-45f6-9cc1-8f113165762f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:10.058177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffee46-3c90-40cd-9a5c-9b8341a25bed None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:10.074408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:10.082075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5908b-e294-4a14-bed5-2e0979c2306a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:10.082734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5908b-e294-4a14-bed5-2e0979c2306a None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:10.083020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5908b-e294-4a14-bed5-2e0979c2306a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:10.083300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5908b-e294-4a14-bed5-2e0979c2306a None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:10.084006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5908b-e294-4a14-bed5-2e0979c2306a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:10.085488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5908b-e294-4a14-bed5-2e0979c2306a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:10.091583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690e0bc-c397-493a-9ef8-dd638494c9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:10.092232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e0bc-c397-493a-9ef8-dd638494c9d2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:10.092558 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e0bc-c397-493a-9ef8-dd638494c9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:10.092841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e0bc-c397-493a-9ef8-dd638494c9d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:10.097265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690e0bc-c397-493a-9ef8-dd638494c9d2 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.domain.id=default, user.id=0840c2e542b841edbc74432280137f72) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:10.097265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690e0bc-c397-493a-9ef8-dd638494c9d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:10.097265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690e0bc-c397-493a-9ef8-dd638494c9d2 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:10.104333 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 24/56] 10.222.0.23 () {56 vars in 1322 bytes} [Mon May 23 15:36:09 2022] GET /identity/v3/role_assignments?user.id=0840c2e542b841edbc74432280137f72&scope.domain.id=default&role.id=04c11dea81274a69974ad5c639f2c60b => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:10.312713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 33/57] 10.222.0.23 () {60 vars in 1140 bytes} [Mon May 23 15:36:09 2022] POST /identity/v3/projects => generated 298 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:36:12.478624 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d262a9e-406c-4f48-9a14-bdbcb78af440 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:12.479145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d262a9e-406c-4f48-9a14-bdbcb78af440 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:12.479247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d262a9e-406c-4f48-9a14-bdbcb78af440 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:12.480565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 25/58] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:12.486182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519d35f-345d-41ef-8204-e5d57a27288a None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:12.486541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519d35f-345d-41ef-8204-e5d57a27288a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:12.486771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519d35f-345d-41ef-8204-e5d57a27288a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:12.504500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-8519d35f-345d-41ef-8204-e5d57a27288a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:12.505678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8519d35f-345d-41ef-8204-e5d57a27288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:12.573008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 34/59] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:12.596658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d262a9e-406c-4f48-9a14-bdbcb78af440 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:12.695146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d262a9e-406c-4f48-9a14-bdbcb78af440 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:12.727211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4208e-63c3-4334-af22-0ccc44b0b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:12.728000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4208e-63c3-4334-af22-0ccc44b0b596 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:12.728288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4208e-63c3-4334-af22-0ccc44b0b596 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:12.728603 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4208e-63c3-4334-af22-0ccc44b0b596 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:12.729776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4208e-63c3-4334-af22-0ccc44b0b596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:12.731224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4208e-63c3-4334-af22-0ccc44b0b596 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:12.733392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 26/60] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:12.738614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519d35f-345d-41ef-8204-e5d57a27288a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:12.810071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:12.833959 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519d35f-345d-41ef-8204-e5d57a27288a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:12.867797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992eaa0f-b7ba-4d5d-881e-7567c6b82bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:12.868503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992eaa0f-b7ba-4d5d-881e-7567c6b82bd5 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:12.868796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992eaa0f-b7ba-4d5d-881e-7567c6b82bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:12.869079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992eaa0f-b7ba-4d5d-881e-7567c6b82bd5 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:12.869802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992eaa0f-b7ba-4d5d-881e-7567c6b82bd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:12.871257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992eaa0f-b7ba-4d5d-881e-7567c6b82bd5 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:12.949160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 35/61] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:36:12 2022] POST /identity/v3/users => generated 275 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:16.332417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82398cd-3a99-4b13-b256-9f6e6bebc167 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.332965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82398cd-3a99-4b13-b256-9f6e6bebc167 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.332965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82398cd-3a99-4b13-b256-9f6e6bebc167 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.334400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 27/62] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:16.336428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfc203-f4dc-41c4-bc43-472dd90aef2b None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.336828 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfc203-f4dc-41c4-bc43-472dd90aef2b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.336992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfc203-f4dc-41c4-bc43-472dd90aef2b None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.338490 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 36/63] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:16.344987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d58eb4-f431-4938-af8d-0b603e465f5a None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.345505 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d58eb4-f431-4938-af8d-0b603e465f5a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.345939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d58eb4-f431-4938-af8d-0b603e465f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.354264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49351c-71a3-499f-9c12-cb7b7adf931c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.359611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49351c-71a3-499f-9c12-cb7b7adf931c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.360144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49351c-71a3-499f-9c12-cb7b7adf931c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.371579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-f9d58eb4-f431-4938-af8d-0b603e465f5a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:16.377586 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f9d58eb4-f431-4938-af8d-0b603e465f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:16.388843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-da49351c-71a3-499f-9c12-cb7b7adf931c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:16.390265 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-da49351c-71a3-499f-9c12-cb7b7adf931c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:16.428394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:16.475524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 28/64] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:16.504083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 37/65] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:16.507856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d58eb4-f431-4938-af8d-0b603e465f5a None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:16.527611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49351c-71a3-499f-9c12-cb7b7adf931c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:16.614413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:16.647614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d58eb4-f431-4938-af8d-0b603e465f5a None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:16.676135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49351c-71a3-499f-9c12-cb7b7adf931c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:16.690827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1d061-40d0-41b6-946a-9c781142ef2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:16.691915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1d061-40d0-41b6-946a-9c781142ef2c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.692459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1d061-40d0-41b6-946a-9c781142ef2c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.692881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1d061-40d0-41b6-946a-9c781142ef2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.705073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1d061-40d0-41b6-946a-9c781142ef2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-8ae1d061-40d0-41b6-946a-9c781142ef2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:16.713434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:16.714472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:16.715245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:16.717528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 29/66] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:36:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:16.720121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d46e8-b228-464b-8021-d87900fa2230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:16.721125 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d46e8-b228-464b-8021-d87900fa2230 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.721539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d46e8-b228-464b-8021-d87900fa2230 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.721943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d46e8-b228-464b-8021-d87900fa2230 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.723611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74aff4d-4a38-4e21-9f3e-7ce6b9dd1ce6 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.724182 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74aff4d-4a38-4e21-9f3e-7ce6b9dd1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.724793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74aff4d-4a38-4e21-9f3e-7ce6b9dd1ce6 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.726307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 30/67] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:16.727185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74aff4d-4a38-4e21-9f3e-7ce6b9dd1ce6 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:16.739112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d46e8-b228-464b-8021-d87900fa2230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-674d46e8-b228-464b-8021-d87900fa2230 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:16.748834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:16.749768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:16.750512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:16.752869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 38/68] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:16.754362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a426a7-c4af-4403-bbd2-9523b081cdf6 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.754851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a426a7-c4af-4403-bbd2-9523b081cdf6 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.755345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a426a7-c4af-4403-bbd2-9523b081cdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.773794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-82a426a7-c4af-4403-bbd2-9523b081cdf6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:16.775275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-82a426a7-c4af-4403-bbd2-9523b081cdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:16.784447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:16.846593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 39/69] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:16.847570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a426a7-c4af-4403-bbd2-9523b081cdf6 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:16.861154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74aff4d-4a38-4e21-9f3e-7ce6b9dd1ce6 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:16.897655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c91500-050f-4024-bbb7-71d32cca4e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:16.898244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c91500-050f-4024-bbb7-71d32cca4e54 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.898532 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c91500-050f-4024-bbb7-71d32cca4e54 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.898811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c91500-050f-4024-bbb7-71d32cca4e54 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.899580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c91500-050f-4024-bbb7-71d32cca4e54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:16.899872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c91500-050f-4024-bbb7-71d32cca4e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:16.901291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c91500-050f-4024-bbb7-71d32cca4e54 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:16.905755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:16.912925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 31/70] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:16.920913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3c6e-ac84-4089-a3cb-b2f63b08a90c None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.920913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3c6e-ac84-4089-a3cb-b2f63b08a90c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.920913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3c6e-ac84-4089-a3cb-b2f63b08a90c None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.920913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 32/71] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:16.920913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7e788-12fc-4609-842e-0d2ce7f33d57 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.920913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7e788-12fc-4609-842e-0d2ce7f33d57 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.921287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7e788-12fc-4609-842e-0d2ce7f33d57 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.921287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 33/72] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:16.921287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7e788-12fc-4609-842e-0d2ce7f33d57 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:16.959988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a426a7-c4af-4403-bbd2-9523b081cdf6 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:16.992744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5bd5d-28b8-4a89-96c7-3fab86d77a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:16.993410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5bd5d-28b8-4a89-96c7-3fab86d77a41 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:16.993695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5bd5d-28b8-4a89-96c7-3fab86d77a41 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:16.993971 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5bd5d-28b8-4a89-96c7-3fab86d77a41 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:16.994722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5bd5d-28b8-4a89-96c7-3fab86d77a41 None admin] RBAC: Adding query filter params (name=member) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:16.995033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5bd5d-28b8-4a89-96c7-3fab86d77a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:16.996442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5bd5d-28b8-4a89-96c7-3fab86d77a41 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:17.004961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 40/73] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:17.005897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5bd5d-28b8-4a89-96c7-3fab86d77a41 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.022285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:17.026557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7e788-12fc-4609-842e-0d2ce7f33d57 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:17.061150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ea8cd-d6f6-44c9-b911-129945bb8e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:17.061955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea8cd-d6f6-44c9-b911-129945bb8e95 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.062234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea8cd-d6f6-44c9-b911-129945bb8e95 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.062512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ea8cd-d6f6-44c9-b911-129945bb8e95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.072799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ea8cd-d6f6-44c9-b911-129945bb8e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-ef2ea8cd-d6f6-44c9-b911-129945bb8e95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:17.080925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:17.081935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:17.082886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 34/74] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:36:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:17.083693 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b747-e19b-478c-9658-eee8b0ccabc2 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.083993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b747-e19b-478c-9658-eee8b0ccabc2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.084282 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935b747-e19b-478c-9658-eee8b0ccabc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.100195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9935b747-e19b-478c-9658-eee8b0ccabc2 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:17.101435 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9935b747-e19b-478c-9658-eee8b0ccabc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:17.106276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5bd5d-28b8-4a89-96c7-3fab86d77a41 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:17.119262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:17.140386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051813e-1aed-4976-9f28-4320f9f45d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:17.141263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051813e-1aed-4976-9f28-4320f9f45d1e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.141555 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051813e-1aed-4976-9f28-4320f9f45d1e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.141836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051813e-1aed-4976-9f28-4320f9f45d1e None admin] PATH_INFO: `/v3/users/0840c2e542b841edbc74432280137f72` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.143063 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051813e-1aed-4976-9f28-4320f9f45d1e None admin] RBAC: Authorizing `identity:get_user(user_id=0840c2e542b841edbc74432280137f72)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:17.144554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051813e-1aed-4976-9f28-4320f9f45d1e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:17.146170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 41/75] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:17 2022] GET /identity/v3/users/0840c2e542b841edbc74432280137f72 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:17.148444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12c0bb-bbe1-4412-8bd8-aaf2e43b3ed0 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.148444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12c0bb-bbe1-4412-8bd8-aaf2e43b3ed0 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.148444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12c0bb-bbe1-4412-8bd8-aaf2e43b3ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.164892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-8e12c0bb-bbe1-4412-8bd8-aaf2e43b3ed0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:17.166256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8e12c0bb-bbe1-4412-8bd8-aaf2e43b3ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:17.173287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 35/76] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:17.174122 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935b747-e19b-478c-9658-eee8b0ccabc2 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.231406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 42/77] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:17.232836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12c0bb-bbe1-4412-8bd8-aaf2e43b3ed0 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.281790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:17.291667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935b747-e19b-478c-9658-eee8b0ccabc2 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:17.326108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5765e-e961-4392-8538-b654d0d5a811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:17.326990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5765e-e961-4392-8538-b654d0d5a811 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.327273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5765e-e961-4392-8538-b654d0d5a811 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.327556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5765e-e961-4392-8538-b654d0d5a811 None admin] PATH_INFO: `/v3/users/7384fa3396274b4e8aa20b57838b11a6` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.337769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12c0bb-bbe1-4412-8bd8-aaf2e43b3ed0 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:17.342436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5765e-e961-4392-8538-b654d0d5a811 None admin] RBAC: Authorizing `identity:get_user(user_id=7384fa3396274b4e8aa20b57838b11a6)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:17.343894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5765e-e961-4392-8538-b654d0d5a811 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:17.345648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 36/78] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:17 2022] GET /identity/v3/users/7384fa3396274b4e8aa20b57838b11a6 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:17.346425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5765e-e961-4392-8538-b654d0d5a811 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.349085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:17.372140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf0b7b-01c0-4357-8170-b59184f41c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:17.372882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf0b7b-01c0-4357-8170-b59184f41c69 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.373169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf0b7b-01c0-4357-8170-b59184f41c69 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.373448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf0b7b-01c0-4357-8170-b59184f41c69 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.374199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf0b7b-01c0-4357-8170-b59184f41c69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:17.374513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf0b7b-01c0-4357-8170-b59184f41c69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:17.375891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf0b7b-01c0-4357-8170-b59184f41c69 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:17.384647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 43/79] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:36:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:17.385428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf0b7b-01c0-4357-8170-b59184f41c69 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.446162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5765e-e961-4392-8538-b654d0d5a811 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:17.479327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb811246-6723-43f1-9e38-a695c8fed020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:17.479991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb811246-6723-43f1-9e38-a695c8fed020 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.480287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb811246-6723-43f1-9e38-a695c8fed020 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.480610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb811246-6723-43f1-9e38-a695c8fed020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.483375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf0b7b-01c0-4357-8170-b59184f41c69 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:17.500641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:17.516662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b828cc-3531-47e4-917b-1007a0264480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:17.517463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b828cc-3531-47e4-917b-1007a0264480 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.517751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b828cc-3531-47e4-917b-1007a0264480 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.518036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b828cc-3531-47e4-917b-1007a0264480 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.527896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b828cc-3531-47e4-917b-1007a0264480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-e8b828cc-3531-47e4-917b-1007a0264480 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:17.536023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:17.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:17.537735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 44/80] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:17.538417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b828cc-3531-47e4-917b-1007a0264480 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.552078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb811246-6723-43f1-9e38-a695c8fed020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:17.553628 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb811246-6723-43f1-9e38-a695c8fed020 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:17.572049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:17.641148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b828cc-3531-47e4-917b-1007a0264480 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:17.651532 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 37/81] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:17.652395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb811246-6723-43f1-9e38-a695c8fed020 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.674767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb3d34-00a9-49ef-82ee-5b1a800abfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:17.675597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3d34-00a9-49ef-82ee-5b1a800abfac None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.675888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3d34-00a9-49ef-82ee-5b1a800abfac None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.676174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3d34-00a9-49ef-82ee-5b1a800abfac None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.677411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb3d34-00a9-49ef-82ee-5b1a800abfac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:17.678884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb3d34-00a9-49ef-82ee-5b1a800abfac None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:17.680520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 45/82] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:17.681291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb3d34-00a9-49ef-82ee-5b1a800abfac None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.706233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:17.757464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb811246-6723-43f1-9e38-a695c8fed020 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:17.775687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:17.779279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb3d34-00a9-49ef-82ee-5b1a800abfac None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:17.791079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da844f7-871a-4442-bb53-28907e1faf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:17.792315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da844f7-871a-4442-bb53-28907e1faf78 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.792315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da844f7-871a-4442-bb53-28907e1faf78 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.792315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da844f7-871a-4442-bb53-28907e1faf78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.812403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72b1be-9383-4624-a3f2-8320df2309a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:17.813283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72b1be-9383-4624-a3f2-8320df2309a7 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.813567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72b1be-9383-4624-a3f2-8320df2309a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.813843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72b1be-9383-4624-a3f2-8320df2309a7 None admin] PATH_INFO: `/v3/users/7384fa3396274b4e8aa20b57838b11a6` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.815026 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72b1be-9383-4624-a3f2-8320df2309a7 None admin] RBAC: Authorizing `identity:get_user(user_id=7384fa3396274b4e8aa20b57838b11a6)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:17.816504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72b1be-9383-4624-a3f2-8320df2309a7 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:17.818100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 46/83] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:17 2022] GET /identity/v3/users/7384fa3396274b4e8aa20b57838b11a6 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:17.818857 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72b1be-9383-4624-a3f2-8320df2309a7 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.863338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da844f7-871a-4442-bb53-28907e1faf78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:17.864870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da844f7-871a-4442-bb53-28907e1faf78 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:17.911103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:17.914729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72b1be-9383-4624-a3f2-8320df2309a7 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:17.947824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f204f05-04a1-4df9-a7cf-3f8416287905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:17.948521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f204f05-04a1-4df9-a7cf-3f8416287905 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:17.948819 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f204f05-04a1-4df9-a7cf-3f8416287905 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:17.949098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f204f05-04a1-4df9-a7cf-3f8416287905 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:17.949832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f204f05-04a1-4df9-a7cf-3f8416287905 None admin] RBAC: Adding query filter params (name=member) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:17.950142 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f204f05-04a1-4df9-a7cf-3f8416287905 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:17.952127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f204f05-04a1-4df9-a7cf-3f8416287905 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:17.960220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 47/84] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:17.960981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f204f05-04a1-4df9-a7cf-3f8416287905 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.966022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 38/85] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:17.966022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da844f7-871a-4442-bb53-28907e1faf78 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:17.992948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:18.061059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da844f7-871a-4442-bb53-28907e1faf78 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:18.062334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f204f05-04a1-4df9-a7cf-3f8416287905 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:18.094395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8449a-26f4-4a25-9ff2-4742fdbb11fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:18.095068 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8449a-26f4-4a25-9ff2-4742fdbb11fd None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:18.095362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8449a-26f4-4a25-9ff2-4742fdbb11fd None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:18.095648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8449a-26f4-4a25-9ff2-4742fdbb11fd None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:18.095908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e776eb0-7d3c-41c6-868c-331573342a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:18.096385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f8449a-26f4-4a25-9ff2-4742fdbb11fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:18.096817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e776eb0-7d3c-41c6-868c-331573342a40 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:18.097105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e776eb0-7d3c-41c6-868c-331573342a40 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:18.097385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e776eb0-7d3c-41c6-868c-331573342a40 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:18.097835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f8449a-26f4-4a25-9ff2-4742fdbb11fd None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:18.098563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e776eb0-7d3c-41c6-868c-331573342a40 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:18.100012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e776eb0-7d3c-41c6-868c-331573342a40 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:18.101747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 48/86] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:17 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:18.102506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e776eb0-7d3c-41c6-868c-331573342a40 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:18.195298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:18.198973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e776eb0-7d3c-41c6-868c-331573342a40 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:18.236474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 39/87] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:36:17 2022] POST /identity/v3/users => generated 289 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:18.236474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8449a-26f4-4a25-9ff2-4742fdbb11fd None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:18.236745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d37b4-3454-4f23-b7de-95739158bcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:18.236745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d37b4-3454-4f23-b7de-95739158bcf5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:18.236745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d37b4-3454-4f23-b7de-95739158bcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:18.236745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d37b4-3454-4f23-b7de-95739158bcf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:18.245330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:18.320919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d37b4-3454-4f23-b7de-95739158bcf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:18.322657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d37b4-3454-4f23-b7de-95739158bcf5 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:18.368879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8449a-26f4-4a25-9ff2-4742fdbb11fd None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:18.411728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170b390-14e6-4eb7-a840-f71e042c84e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:18.412707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170b390-14e6-4eb7-a840-f71e042c84e2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:18.412994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170b390-14e6-4eb7-a840-f71e042c84e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:18.413595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170b390-14e6-4eb7-a840-f71e042c84e2 None admin] PATH_INFO: `/v3/users/7384fa3396274b4e8aa20b57838b11a6` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:18.414972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170b390-14e6-4eb7-a840-f71e042c84e2 None admin] RBAC: Authorizing `identity:get_user(user_id=7384fa3396274b4e8aa20b57838b11a6)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:18.415954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170b390-14e6-4eb7-a840-f71e042c84e2 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:18.417626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 40/88] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:18 2022] GET /identity/v3/users/7384fa3396274b4e8aa20b57838b11a6 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:18.419741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170b390-14e6-4eb7-a840-f71e042c84e2 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:18.433566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:18.439340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 49/89] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:18.440435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d37b4-3454-4f23-b7de-95739158bcf5 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:18.502489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:18.547224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170b390-14e6-4eb7-a840-f71e042c84e2 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:18.575665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d37b4-3454-4f23-b7de-95739158bcf5 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:18.593850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e1e2-805b-4a2a-bd7c-d340e5d4ab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:18.595370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e1e2-805b-4a2a-bd7c-d340e5d4ab13 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:18.595671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e1e2-805b-4a2a-bd7c-d340e5d4ab13 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:18.595957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75e1e2-805b-4a2a-bd7c-d340e5d4ab13 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:18.597251 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75e1e2-805b-4a2a-bd7c-d340e5d4ab13 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:18.599413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75e1e2-805b-4a2a-bd7c-d340e5d4ab13 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:18.601608 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 41/90] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:18 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:18.603133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e1e2-805b-4a2a-bd7c-d340e5d4ab13 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:18.617221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55df66b-06fc-4b17-9de9-4c8572882b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:18.617895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55df66b-06fc-4b17-9de9-4c8572882b91 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:18.618191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55df66b-06fc-4b17-9de9-4c8572882b91 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:18.618477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55df66b-06fc-4b17-9de9-4c8572882b91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:18.619328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55df66b-06fc-4b17-9de9-4c8572882b91 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=9811a47b7d5449f999b0f971ee26f008, user.id=0840c2e542b841edbc74432280137f72) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:18.619849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55df66b-06fc-4b17-9de9-4c8572882b91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:18.621351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55df66b-06fc-4b17-9de9-4c8572882b91 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:18.627910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 50/91] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:18 2022] GET /identity/v3/role_assignments?user.id=0840c2e542b841edbc74432280137f72&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=04c11dea81274a69974ad5c639f2c60b => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:18.628779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55df66b-06fc-4b17-9de9-4c8572882b91 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:18.741855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:18.812991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75e1e2-805b-4a2a-bd7c-d340e5d4ab13 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:18.833520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55df66b-06fc-4b17-9de9-4c8572882b91 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:18.871197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055120ae-2d50-4533-943c-9409ad05736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:18.872232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055120ae-2d50-4533-943c-9409ad05736c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:18.872669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055120ae-2d50-4533-943c-9409ad05736c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:18.873057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055120ae-2d50-4533-943c-9409ad05736c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:18.883789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b4ae6-a415-4011-9946-369b994e204c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:18.884953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4ae6-a415-4011-9946-369b994e204c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:18.885382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4ae6-a415-4011-9946-369b994e204c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:18.885803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4ae6-a415-4011-9946-369b994e204c None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:18.887893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b4ae6-a415-4011-9946-369b994e204c None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:18.889648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b4ae6-a415-4011-9946-369b994e204c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:18.891760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 51/92] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:18 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:18.892691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b4ae6-a415-4011-9946-369b994e204c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:18.895429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:19.020247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055120ae-2d50-4533-943c-9409ad05736c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:19.022811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055120ae-2d50-4533-943c-9409ad05736c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:19.056296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b4ae6-a415-4011-9946-369b994e204c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:19.102742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:19.117139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e73f6-722d-49c6-b48e-7b397071fcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:19.118000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e73f6-722d-49c6-b48e-7b397071fcc5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:19.118422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e73f6-722d-49c6-b48e-7b397071fcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:19.118835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e73f6-722d-49c6-b48e-7b397071fcc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:19.119805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e73f6-722d-49c6-b48e-7b397071fcc5 None admin] RBAC: Adding query filter params (role.id=177939861d6e4e82b48401be184d66b4, scope.project.id=9811a47b7d5449f999b0f971ee26f008, user.id=7384fa3396274b4e8aa20b57838b11a6) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:19.120345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e73f6-722d-49c6-b48e-7b397071fcc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:19.121995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e73f6-722d-49c6-b48e-7b397071fcc5 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:19.132684 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 52/93] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:18 2022] GET /identity/v3/role_assignments?user.id=7384fa3396274b4e8aa20b57838b11a6&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=177939861d6e4e82b48401be184d66b4 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:19.134244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e73f6-722d-49c6-b48e-7b397071fcc5 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:19.250062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 42/94] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:19.257959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055120ae-2d50-4533-943c-9409ad05736c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:19.307776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:19.359484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e73f6-722d-49c6-b48e-7b397071fcc5 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:19.374555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055120ae-2d50-4533-943c-9409ad05736c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:19.410658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836e7d2-b18a-4577-b04d-49f11b3ae938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:19.411348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836e7d2-b18a-4577-b04d-49f11b3ae938 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:19.411640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836e7d2-b18a-4577-b04d-49f11b3ae938 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:19.411925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836e7d2-b18a-4577-b04d-49f11b3ae938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:19.416463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836e7d2-b18a-4577-b04d-49f11b3ae938 None admin] RBAC: Adding query filter params (role.id=f7e0a506c7894b20b2b65fedfd2244e5, scope.project.id=9811a47b7d5449f999b0f971ee26f008, user.id=7384fa3396274b4e8aa20b57838b11a6) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:19.416463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836e7d2-b18a-4577-b04d-49f11b3ae938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:19.416463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836e7d2-b18a-4577-b04d-49f11b3ae938 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:19.418403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7342265-20ce-4907-b42d-1292aa9ed708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:19.419984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7342265-20ce-4907-b42d-1292aa9ed708 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:19.420438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7342265-20ce-4907-b42d-1292aa9ed708 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:19.420853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7342265-20ce-4907-b42d-1292aa9ed708 None admin] PATH_INFO: `/v3/projects/da738cb1cdc344cb8f742cc1f27f070c` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:19.421424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 53/95] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:19 2022] GET /identity/v3/role_assignments?user.id=7384fa3396274b4e8aa20b57838b11a6&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=f7e0a506c7894b20b2b65fedfd2244e5 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:19.424382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7342265-20ce-4907-b42d-1292aa9ed708 None admin] RBAC: Authorizing `identity:get_project(project_id=da738cb1cdc344cb8f742cc1f27f070c)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:19.424382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7342265-20ce-4907-b42d-1292aa9ed708 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:19.425830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 43/96] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:19 2022] GET /identity/v3/projects/da738cb1cdc344cb8f742cc1f27f070c => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:19.432507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836e7d2-b18a-4577-b04d-49f11b3ae938 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:19.498468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:19.543801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836e7d2-b18a-4577-b04d-49f11b3ae938 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:19.581253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce961f0d-6570-4b26-b2b9-ab785e06a924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:19.581946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce961f0d-6570-4b26-b2b9-ab785e06a924 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:19.582238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce961f0d-6570-4b26-b2b9-ab785e06a924 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:19.582521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce961f0d-6570-4b26-b2b9-ab785e06a924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:19.583391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce961f0d-6570-4b26-b2b9-ab785e06a924 None admin] RBAC: Adding query filter params (role.id=177939861d6e4e82b48401be184d66b4, scope.project.id=da738cb1cdc344cb8f742cc1f27f070c, user.id=7384fa3396274b4e8aa20b57838b11a6) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:19.583714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce961f0d-6570-4b26-b2b9-ab785e06a924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:19.585233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce961f0d-6570-4b26-b2b9-ab785e06a924 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:19.592608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 54/97] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:19 2022] GET /identity/v3/role_assignments?user.id=7384fa3396274b4e8aa20b57838b11a6&scope.project.id=da738cb1cdc344cb8f742cc1f27f070c&role.id=177939861d6e4e82b48401be184d66b4 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:20.949751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53272d-801f-4efd-88d0-8996ccdd59e3 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:20.949751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53272d-801f-4efd-88d0-8996ccdd59e3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:20.949751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53272d-801f-4efd-88d0-8996ccdd59e3 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:20.949751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 44/98] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:20.955339 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cd3e9-5cf1-4546-88a0-3c065b59b906 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:20.956209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cd3e9-5cf1-4546-88a0-3c065b59b906 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:20.956719 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cd3e9-5cf1-4546-88a0-3c065b59b906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:20.980502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-c80cd3e9-5cf1-4546-88a0-3c065b59b906 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:20.982601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c80cd3e9-5cf1-4546-88a0-3c065b59b906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:21.074741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 55/99] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:21.129266 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53272d-801f-4efd-88d0-8996ccdd59e3 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:21.351737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53272d-801f-4efd-88d0-8996ccdd59e3 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:21.362917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:21.396150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828d1e3-8aed-4ff3-bed0-b0e203cf829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:21.397012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828d1e3-8aed-4ff3-bed0-b0e203cf829b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:21.397300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828d1e3-8aed-4ff3-bed0-b0e203cf829b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:21.397581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828d1e3-8aed-4ff3-bed0-b0e203cf829b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:21.412088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828d1e3-8aed-4ff3-bed0-b0e203cf829b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-5828d1e3-8aed-4ff3-bed0-b0e203cf829b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:21.419978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:21.420847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:21.421596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 45/100] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:21.428301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cd3e9-5cf1-4546-88a0-3c065b59b906 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:21.483919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:21.531289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cd3e9-5cf1-4546-88a0-3c065b59b906 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:21.575926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f780f8-f1e4-4032-87f8-32c75e8bb1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:21.576560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f780f8-f1e4-4032-87f8-32c75e8bb1e4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:21.576855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f780f8-f1e4-4032-87f8-32c75e8bb1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:21.577139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f780f8-f1e4-4032-87f8-32c75e8bb1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:21.577900 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f780f8-f1e4-4032-87f8-32c75e8bb1e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:21.578231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f780f8-f1e4-4032-87f8-32c75e8bb1e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:21.579609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f780f8-f1e4-4032-87f8-32c75e8bb1e4 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:21.588298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 56/101] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:21.594113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828d1e3-8aed-4ff3-bed0-b0e203cf829b None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:21.625803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24bf6a-0a32-483d-8269-a1b6b962a270 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:21.626101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24bf6a-0a32-483d-8269-a1b6b962a270 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:21.626375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24bf6a-0a32-483d-8269-a1b6b962a270 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:21.627885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 57/102] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:21.632638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dc856-5f26-49df-90af-d2efcd68fd74 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:21.632930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dc856-5f26-49df-90af-d2efcd68fd74 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:21.633456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dc856-5f26-49df-90af-d2efcd68fd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:21.662003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-521dc856-5f26-49df-90af-d2efcd68fd74 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:21.663183 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-521dc856-5f26-49df-90af-d2efcd68fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:21.802369 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828d1e3-8aed-4ff3-bed0-b0e203cf829b None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:21.835510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 58/103] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:21.850275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976f1c5-be78-4861-934f-e4fd0a9ce058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:21.851164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976f1c5-be78-4861-934f-e4fd0a9ce058 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:21.851517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976f1c5-be78-4861-934f-e4fd0a9ce058 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:21.851812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976f1c5-be78-4861-934f-e4fd0a9ce058 None admin] PATH_INFO: `/v3/users/10af1686119142b1a26f7b138e6faaeb` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:21.867017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dc856-5f26-49df-90af-d2efcd68fd74 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:21.897732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976f1c5-be78-4861-934f-e4fd0a9ce058 None admin] RBAC: Authorizing `identity:get_user(user_id=10af1686119142b1a26f7b138e6faaeb)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:21.899232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976f1c5-be78-4861-934f-e4fd0a9ce058 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:21.901012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 46/104] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:21 2022] GET /identity/v3/users/10af1686119142b1a26f7b138e6faaeb => generated 289 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:21.907612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976f1c5-be78-4861-934f-e4fd0a9ce058 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:21.974134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:21.975060 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:22.052600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976f1c5-be78-4861-934f-e4fd0a9ce058 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:22.095822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad4ece-9e00-440e-9c48-9d5ccf6e781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:22.096995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad4ece-9e00-440e-9c48-9d5ccf6e781d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.098487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad4ece-9e00-440e-9c48-9d5ccf6e781d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.098961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad4ece-9e00-440e-9c48-9d5ccf6e781d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.114158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dc856-5f26-49df-90af-d2efcd68fd74 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:22.185535 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222da08-624c-4207-94a8-2546d295c385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:22.186684 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222da08-624c-4207-94a8-2546d295c385 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.187164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222da08-624c-4207-94a8-2546d295c385 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.188744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222da08-624c-4207-94a8-2546d295c385 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.213433 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222da08-624c-4207-94a8-2546d295c385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-e222da08-624c-4207-94a8-2546d295c385 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:22.222010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:22.222830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:22.223582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:22.236656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 59/105] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:36:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:22.238999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b8b8c-d773-4dbb-ab79-29fd5e16fd14 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.239511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b8b8c-d773-4dbb-ab79-29fd5e16fd14 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.239996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b8b8c-d773-4dbb-ab79-29fd5e16fd14 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.241508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 60/106] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:22.242542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b8b8c-d773-4dbb-ab79-29fd5e16fd14 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:22.242646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad4ece-9e00-440e-9c48-9d5ccf6e781d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:22.244248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad4ece-9e00-440e-9c48-9d5ccf6e781d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:22.315330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:22.401921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 47/107] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:22.408383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8f2c4-60de-4aa0-9d32-24038b1f2e04 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.409248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8f2c4-60de-4aa0-9d32-24038b1f2e04 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.409737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8f2c4-60de-4aa0-9d32-24038b1f2e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.436389 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b8b8c-d773-4dbb-ab79-29fd5e16fd14 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:22.438244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:22.439548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-98e8f2c4-60de-4aa0-9d32-24038b1f2e04 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:22.441058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-98e8f2c4-60de-4aa0-9d32-24038b1f2e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:22.467363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf99daf-8c0c-4181-8966-6fcb4c22d6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:22.468055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf99daf-8c0c-4181-8966-6fcb4c22d6b8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.468351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf99daf-8c0c-4181-8966-6fcb4c22d6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.468671 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf99daf-8c0c-4181-8966-6fcb4c22d6b8 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.469720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf99daf-8c0c-4181-8966-6fcb4c22d6b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:22.470043 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf99daf-8c0c-4181-8966-6fcb4c22d6b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:22.471779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf99daf-8c0c-4181-8966-6fcb4c22d6b8 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:22.483549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 61/108] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:22.484720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf99daf-8c0c-4181-8966-6fcb4c22d6b8 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:22.533746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 48/109] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:22.535507 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecfbdc-2ebb-422d-92e6-283aa5550a6b None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.536138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecfbdc-2ebb-422d-92e6-283aa5550a6b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.536631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecfbdc-2ebb-422d-92e6-283aa5550a6b None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.538426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 49/110] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:22.539386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecfbdc-2ebb-422d-92e6-283aa5550a6b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:22.601712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:22.619037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf99daf-8c0c-4181-8966-6fcb4c22d6b8 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:22.651605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecfbdc-2ebb-422d-92e6-283aa5550a6b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:22.662220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea8525-5817-4048-8adc-984029394415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:22.663398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea8525-5817-4048-8adc-984029394415 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.663702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea8525-5817-4048-8adc-984029394415 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.664045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea8525-5817-4048-8adc-984029394415 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.665862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea8525-5817-4048-8adc-984029394415 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:22.667536 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea8525-5817-4048-8adc-984029394415 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:22.670364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 62/111] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:22 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:22.675084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc79fd-52dc-4eec-ab9d-d2ed5f6de669 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.675084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc79fd-52dc-4eec-ab9d-d2ed5f6de669 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.675084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc79fd-52dc-4eec-ab9d-d2ed5f6de669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.691515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-89bc79fd-52dc-4eec-ab9d-d2ed5f6de669 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:22.692757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-89bc79fd-52dc-4eec-ab9d-d2ed5f6de669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:22.695297 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c8f64-3984-414c-bc1e-b9bfe74b4d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:22.696183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c8f64-3984-414c-bc1e-b9bfe74b4d63 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.696510 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c8f64-3984-414c-bc1e-b9bfe74b4d63 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.696793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c8f64-3984-414c-bc1e-b9bfe74b4d63 None admin] PATH_INFO: `/v3/users/0840c2e542b841edbc74432280137f72` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.697985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c8f64-3984-414c-bc1e-b9bfe74b4d63 None admin] RBAC: Authorizing `identity:get_user(user_id=0840c2e542b841edbc74432280137f72)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:22.699423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c8f64-3984-414c-bc1e-b9bfe74b4d63 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:22.705945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 50/112] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:22 2022] GET /identity/v3/users/0840c2e542b841edbc74432280137f72 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:22.705945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c8f64-3984-414c-bc1e-b9bfe74b4d63 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:22.727693 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:22.776416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 63/113] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:22.777301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc79fd-52dc-4eec-ab9d-d2ed5f6de669 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:22.821009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c8f64-3984-414c-bc1e-b9bfe74b4d63 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:22.845477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:22.856565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ad404-a196-4796-b80d-0ff99e689093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:22.857971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ad404-a196-4796-b80d-0ff99e689093 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.858259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ad404-a196-4796-b80d-0ff99e689093 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.858532 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ad404-a196-4796-b80d-0ff99e689093 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.859712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ad404-a196-4796-b80d-0ff99e689093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:22.861301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ad404-a196-4796-b80d-0ff99e689093 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:22.862960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 51/114] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:22.864139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3b60d-225d-4188-a729-e2be690c1802 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.864453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3b60d-225d-4188-a729-e2be690c1802 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.864732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3b60d-225d-4188-a729-e2be690c1802 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.866121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 52/115] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:22.866633 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3b60d-225d-4188-a729-e2be690c1802 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:22.897768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc79fd-52dc-4eec-ab9d-d2ed5f6de669 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:22.934410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a00c5-4a9f-42ef-9d75-c1e7be411b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:22.935579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a00c5-4a9f-42ef-9d75-c1e7be411b1d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:22.935873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a00c5-4a9f-42ef-9d75-c1e7be411b1d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:22.936151 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a00c5-4a9f-42ef-9d75-c1e7be411b1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:22.938767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a00c5-4a9f-42ef-9d75-c1e7be411b1d None admin] RBAC: Adding query filter params (role.id=2a5bf966af5a486fa21c10760ebfbcad, scope.project.id=9811a47b7d5449f999b0f971ee26f008, user.id=10af1686119142b1a26f7b138e6faaeb) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:22.939088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a00c5-4a9f-42ef-9d75-c1e7be411b1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:22.940651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a00c5-4a9f-42ef-9d75-c1e7be411b1d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:22.946036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 64/116] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:22 2022] GET /identity/v3/role_assignments?user.id=10af1686119142b1a26f7b138e6faaeb&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=2a5bf966af5a486fa21c10760ebfbcad => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:22.946818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a00c5-4a9f-42ef-9d75-c1e7be411b1d None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:22.978794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:22.984937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3b60d-225d-4188-a729-e2be690c1802 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:23.023183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba1636-d096-4a40-bf00-ba85db173b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:23.024066 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba1636-d096-4a40-bf00-ba85db173b84 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.024351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba1636-d096-4a40-bf00-ba85db173b84 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.024692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba1636-d096-4a40-bf00-ba85db173b84 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.025996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba1636-d096-4a40-bf00-ba85db173b84 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:23.027489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba1636-d096-4a40-bf00-ba85db173b84 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:23.029612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 53/117] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:22 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:23.030616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390230c-3f34-43bf-91dc-22539560b165 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.030905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390230c-3f34-43bf-91dc-22539560b165 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.031182 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390230c-3f34-43bf-91dc-22539560b165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.050033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8390230c-3f34-43bf-91dc-22539560b165 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:23.051273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8390230c-3f34-43bf-91dc-22539560b165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:23.066137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a00c5-4a9f-42ef-9d75-c1e7be411b1d None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:23.112048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc772b43-273c-4ddd-a4ba-a6b1973f500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:23.112893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc772b43-273c-4ddd-a4ba-a6b1973f500a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.113188 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc772b43-273c-4ddd-a4ba-a6b1973f500a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.113464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc772b43-273c-4ddd-a4ba-a6b1973f500a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.116685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:23.125461 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc772b43-273c-4ddd-a4ba-a6b1973f500a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:23.130073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 54/118] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:23.131298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390230c-3f34-43bf-91dc-22539560b165 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-cc772b43-273c-4ddd-a4ba-a6b1973f500a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:23.133646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:23.134551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:23.135348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:23.140682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 65/119] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:23.142117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd1fb8-f7b0-472e-8c76-b0266d8f0bd5 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.142405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd1fb8-f7b0-472e-8c76-b0266d8f0bd5 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.142884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd1fb8-f7b0-472e-8c76-b0266d8f0bd5 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.144670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 66/120] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:23.146446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd1fb8-f7b0-472e-8c76-b0266d8f0bd5 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:23.242197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:23.254915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd1fb8-f7b0-472e-8c76-b0266d8f0bd5 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:23.258959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390230c-3f34-43bf-91dc-22539560b165 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:23.298200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd60f2-10b1-4cdc-a706-1ebb782fecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:23.298873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd60f2-10b1-4cdc-a706-1ebb782fecdd None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.299157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd60f2-10b1-4cdc-a706-1ebb782fecdd None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.299433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd60f2-10b1-4cdc-a706-1ebb782fecdd None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.300148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd60f2-10b1-4cdc-a706-1ebb782fecdd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:23.301182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452e073-9a45-4ed6-9ead-3903c78967f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:23.301692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd60f2-10b1-4cdc-a706-1ebb782fecdd None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:23.302231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452e073-9a45-4ed6-9ead-3903c78967f4 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.302583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452e073-9a45-4ed6-9ead-3903c78967f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.302923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452e073-9a45-4ed6-9ead-3903c78967f4 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008/users/0840c2e542b841edbc74432280137f72/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.304948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452e073-9a45-4ed6-9ead-3903c78967f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9811a47b7d5449f999b0f971ee26f008, user_id=0840c2e542b841edbc74432280137f72, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:23.306454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452e073-9a45-4ed6-9ead-3903c78967f4 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:23.394939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 67/121] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:36:23 2022] PUT /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008/users/0840c2e542b841edbc74432280137f72/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:23.401023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452e073-9a45-4ed6-9ead-3903c78967f4 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:23.421481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 55/122] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:36:23 2022] POST /identity/v3/projects => generated 302 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:23.430524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8569f59-5f45-437b-8bca-0c3b8a2439c6 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.430894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8569f59-5f45-437b-8bca-0c3b8a2439c6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.431289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8569f59-5f45-437b-8bca-0c3b8a2439c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.466392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-d8569f59-5f45-437b-8bca-0c3b8a2439c6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:23.467691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d8569f59-5f45-437b-8bca-0c3b8a2439c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:23.521105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452e073-9a45-4ed6-9ead-3903c78967f4 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:23.532778 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:23.602409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f690ff-3150-4ee9-8c45-01d5c441346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:23.603227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f690ff-3150-4ee9-8c45-01d5c441346b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.603525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f690ff-3150-4ee9-8c45-01d5c441346b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.603809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f690ff-3150-4ee9-8c45-01d5c441346b None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.604664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 56/123] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:23.604841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f690ff-3150-4ee9-8c45-01d5c441346b None admin] RBAC: Adding query filter params (name=member) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:23.605027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f690ff-3150-4ee9-8c45-01d5c441346b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:23.606153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8569f59-5f45-437b-8bca-0c3b8a2439c6 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:23.606468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f690ff-3150-4ee9-8c45-01d5c441346b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:23.626952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 68/124] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:23.627933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f690ff-3150-4ee9-8c45-01d5c441346b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:23.650438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:23.721632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8569f59-5f45-437b-8bca-0c3b8a2439c6 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:23.758697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdd1fb-052e-417b-8d0c-d8541f909882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:23.759930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdd1fb-052e-417b-8d0c-d8541f909882 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.760443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdd1fb-052e-417b-8d0c-d8541f909882 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.760915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdd1fb-052e-417b-8d0c-d8541f909882 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.761396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f690ff-3150-4ee9-8c45-01d5c441346b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:23.772967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdd1fb-052e-417b-8d0c-d8541f909882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-5efdd1fb-052e-417b-8d0c-d8541f909882 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:23.781694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:23.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:23.783319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:23.786226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 57/125] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:23.786644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdd1fb-052e-417b-8d0c-d8541f909882 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:23.803275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cda5d3-4517-410e-9e58-547a79929dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:23.804198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda5d3-4517-410e-9e58-547a79929dcf None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.804567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda5d3-4517-410e-9e58-547a79929dcf None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.804848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cda5d3-4517-410e-9e58-547a79929dcf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.817219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cda5d3-4517-410e-9e58-547a79929dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:23.824909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-d1cda5d3-4517-410e-9e58-547a79929dcf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:23.831701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:23.837213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 69/126] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:36:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:23.837985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cda5d3-4517-410e-9e58-547a79929dcf None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:23.907839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdd1fb-052e-417b-8d0c-d8541f909882 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:23.916983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:23.944923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef19d5-3cd2-4caf-8a87-d8e2ba063106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:23.945872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef19d5-3cd2-4caf-8a87-d8e2ba063106 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.946161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef19d5-3cd2-4caf-8a87-d8e2ba063106 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.946448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef19d5-3cd2-4caf-8a87-d8e2ba063106 None admin] PATH_INFO: `/v3/users/7384fa3396274b4e8aa20b57838b11a6` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.947779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef19d5-3cd2-4caf-8a87-d8e2ba063106 None admin] RBAC: Authorizing `identity:get_user(user_id=7384fa3396274b4e8aa20b57838b11a6)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:23.949318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef19d5-3cd2-4caf-8a87-d8e2ba063106 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:23.952775 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 58/127] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:23 2022] GET /identity/v3/users/7384fa3396274b4e8aa20b57838b11a6 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:23.953675 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef19d5-3cd2-4caf-8a87-d8e2ba063106 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:23.955367 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cda5d3-4517-410e-9e58-547a79929dcf None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:23.992305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09c6d4-80d6-44d5-868c-544e4714fe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:23.993767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09c6d4-80d6-44d5-868c-544e4714fe9b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:23.994227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09c6d4-80d6-44d5-868c-544e4714fe9b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:23.994713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09c6d4-80d6-44d5-868c-544e4714fe9b None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:23.995610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09c6d4-80d6-44d5-868c-544e4714fe9b None admin] RBAC: Adding query filter params (name=member) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:23.996072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09c6d4-80d6-44d5-868c-544e4714fe9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:23.998400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09c6d4-80d6-44d5-868c-544e4714fe9b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:24.008082 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 70/128] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:24.009421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09c6d4-80d6-44d5-868c-544e4714fe9b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:24.059181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:24.070575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef19d5-3cd2-4caf-8a87-d8e2ba063106 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:24.117414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09c6d4-80d6-44d5-868c-544e4714fe9b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:24.140155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ffc2e-6e52-451b-bf6d-d4659a060784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:24.141116 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ffc2e-6e52-451b-bf6d-d4659a060784 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:24.141549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ffc2e-6e52-451b-bf6d-d4659a060784 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:24.141988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ffc2e-6e52-451b-bf6d-d4659a060784 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:24.142927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ffc2e-6e52-451b-bf6d-d4659a060784 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:24.143480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ffc2e-6e52-451b-bf6d-d4659a060784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:24.147919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ffc2e-6e52-451b-bf6d-d4659a060784 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:24.166979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 59/129] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:36:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:24.168081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ffc2e-6e52-451b-bf6d-d4659a060784 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:24.192739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:24.201626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d867d1f-6526-4998-a44b-b7ad7666147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:24.202700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d867d1f-6526-4998-a44b-b7ad7666147b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:24.206931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d867d1f-6526-4998-a44b-b7ad7666147b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:24.206931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d867d1f-6526-4998-a44b-b7ad7666147b None admin] PATH_INFO: `/v3/projects/da738cb1cdc344cb8f742cc1f27f070c` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:24.206931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d867d1f-6526-4998-a44b-b7ad7666147b None admin] RBAC: Authorizing `identity:get_project(project_id=da738cb1cdc344cb8f742cc1f27f070c)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:24.209481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d867d1f-6526-4998-a44b-b7ad7666147b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:24.212704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 71/130] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:24 2022] GET /identity/v3/projects/da738cb1cdc344cb8f742cc1f27f070c => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:24.214454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d867d1f-6526-4998-a44b-b7ad7666147b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:24.368247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ffc2e-6e52-451b-bf6d-d4659a060784 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:24.401950 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d867d1f-6526-4998-a44b-b7ad7666147b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:24.437421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92edd744-75d2-4926-bf16-0052978651eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:24.438302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edd744-75d2-4926-bf16-0052978651eb None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:24.438592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edd744-75d2-4926-bf16-0052978651eb None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:24.438878 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92edd744-75d2-4926-bf16-0052978651eb None admin] PATH_INFO: `/v3/users/7384fa3396274b4e8aa20b57838b11a6` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:24.440133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92edd744-75d2-4926-bf16-0052978651eb None admin] RBAC: Authorizing `identity:get_user(user_id=7384fa3396274b4e8aa20b57838b11a6)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:24.441188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51322c-4bd5-4dbe-b448-90731249ec03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:24.441684 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92edd744-75d2-4926-bf16-0052978651eb None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:24.442082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51322c-4bd5-4dbe-b448-90731249ec03 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:24.442368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51322c-4bd5-4dbe-b448-90731249ec03 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:24.442647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51322c-4bd5-4dbe-b448-90731249ec03 None admin] PATH_INFO: `/v3/users/7384fa3396274b4e8aa20b57838b11a6` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:24.443537 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 72/131] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:24 2022] GET /identity/v3/users/7384fa3396274b4e8aa20b57838b11a6 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:24.444057 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92edd744-75d2-4926-bf16-0052978651eb None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:24.445135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51322c-4bd5-4dbe-b448-90731249ec03 None admin] RBAC: Authorizing `identity:get_user(user_id=7384fa3396274b4e8aa20b57838b11a6)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:24.446567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51322c-4bd5-4dbe-b448-90731249ec03 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:24.448204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 60/132] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:24 2022] GET /identity/v3/users/7384fa3396274b4e8aa20b57838b11a6 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:24.453423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51322c-4bd5-4dbe-b448-90731249ec03 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:24.518661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:24.569631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:24.629735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51322c-4bd5-4dbe-b448-90731249ec03 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:24.633984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92edd744-75d2-4926-bf16-0052978651eb None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:24.674851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d848d-6172-40c2-bfe7-8dc119aab0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:24.675810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d848d-6172-40c2-bfe7-8dc119aab0c0 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:24.676099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d848d-6172-40c2-bfe7-8dc119aab0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:24.676391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d848d-6172-40c2-bfe7-8dc119aab0c0 None admin] PATH_INFO: `/v3/projects/da738cb1cdc344cb8f742cc1f27f070c/users/7384fa3396274b4e8aa20b57838b11a6/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:24.678396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d848d-6172-40c2-bfe7-8dc119aab0c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=da738cb1cdc344cb8f742cc1f27f070c, user_id=7384fa3396274b4e8aa20b57838b11a6, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:24.679859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d848d-6172-40c2-bfe7-8dc119aab0c0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:24.698381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400df67-bf4d-4a2f-b8ef-9824b117d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:24.699412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400df67-bf4d-4a2f-b8ef-9824b117d252 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:24.699793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400df67-bf4d-4a2f-b8ef-9824b117d252 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:24.700084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400df67-bf4d-4a2f-b8ef-9824b117d252 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:24.701548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400df67-bf4d-4a2f-b8ef-9824b117d252 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:24.703048 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400df67-bf4d-4a2f-b8ef-9824b117d252 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:24.705070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 61/133] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:24 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:24.705680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400df67-bf4d-4a2f-b8ef-9824b117d252 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:24.772771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 73/134] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:36:24 2022] PUT /identity/v3/projects/da738cb1cdc344cb8f742cc1f27f070c/users/7384fa3396274b4e8aa20b57838b11a6/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:24.773594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d848d-6172-40c2-bfe7-8dc119aab0c0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:24.833478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:24.837718 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400df67-bf4d-4a2f-b8ef-9824b117d252 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:24.901666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6f696-67a6-4d49-9616-88c9b08bee9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:24.902461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6f696-67a6-4d49-9616-88c9b08bee9c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:24.902750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6f696-67a6-4d49-9616-88c9b08bee9c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:24.903027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6f696-67a6-4d49-9616-88c9b08bee9c None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:24.905576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6f696-67a6-4d49-9616-88c9b08bee9c None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:24.907026 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6f696-67a6-4d49-9616-88c9b08bee9c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:24.908968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 62/135] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:24 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:24.916968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6f696-67a6-4d49-9616-88c9b08bee9c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:24.961888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d848d-6172-40c2-bfe7-8dc119aab0c0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:25.014272 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:25.038059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e611b-b022-46e8-95e1-5471d0674b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:25.039320 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e611b-b022-46e8-95e1-5471d0674b43 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:25.039844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e611b-b022-46e8-95e1-5471d0674b43 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:25.040326 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e611b-b022-46e8-95e1-5471d0674b43 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008/users/7384fa3396274b4e8aa20b57838b11a6/roles/f7e0a506c7894b20b2b65fedfd2244e5` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:25.047620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e611b-b022-46e8-95e1-5471d0674b43 None admin] RBAC: Authorizing `identity:create_grant(project_id=9811a47b7d5449f999b0f971ee26f008, user_id=7384fa3396274b4e8aa20b57838b11a6, role_id=f7e0a506c7894b20b2b65fedfd2244e5)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:25.049409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e611b-b022-46e8-95e1-5471d0674b43 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:25.084539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6f696-67a6-4d49-9616-88c9b08bee9c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:25.128769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c027e7f-a920-4803-a887-cecede4b5319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:25.129673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c027e7f-a920-4803-a887-cecede4b5319 None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:25.130008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c027e7f-a920-4803-a887-cecede4b5319 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:25.130287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c027e7f-a920-4803-a887-cecede4b5319 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008/users/7384fa3396274b4e8aa20b57838b11a6/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:25.132449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c027e7f-a920-4803-a887-cecede4b5319 None admin] RBAC: Authorizing `identity:create_grant(project_id=9811a47b7d5449f999b0f971ee26f008, user_id=7384fa3396274b4e8aa20b57838b11a6, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:25.134309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c027e7f-a920-4803-a887-cecede4b5319 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:25.158751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 74/136] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:36:24 2022] PUT /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008/users/7384fa3396274b4e8aa20b57838b11a6/roles/f7e0a506c7894b20b2b65fedfd2244e5 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:25.243009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 63/137] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:36:24 2022] PUT /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008/users/7384fa3396274b4e8aa20b57838b11a6/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:25.477946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9994a3-9390-4d35-8fef-0b135e42cb8f None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:25.478622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9994a3-9390-4d35-8fef-0b135e42cb8f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:25.479125 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9994a3-9390-4d35-8fef-0b135e42cb8f None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:25.481190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 75/138] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:25.486835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875ee03-0e52-4097-a37a-df3b968c998e None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:25.487133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875ee03-0e52-4097-a37a-df3b968c998e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:25.487412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875ee03-0e52-4097-a37a-df3b968c998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:25.507450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1875ee03-0e52-4097-a37a-df3b968c998e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:25.508986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1875ee03-0e52-4097-a37a-df3b968c998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:25.533521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:25.591348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 64/139] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:25.614659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9994a3-9390-4d35-8fef-0b135e42cb8f None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:25.735745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9994a3-9390-4d35-8fef-0b135e42cb8f None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:25.779556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d204bc-4e45-4055-9c59-7bb980a09af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:25.780387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d204bc-4e45-4055-9c59-7bb980a09af2 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:25.780669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d204bc-4e45-4055-9c59-7bb980a09af2 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:25.780943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d204bc-4e45-4055-9c59-7bb980a09af2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:25.791771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d204bc-4e45-4055-9c59-7bb980a09af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-c0d204bc-4e45-4055-9c59-7bb980a09af2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:25.808938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:25.809848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:25.811647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 76/140] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:25.816480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875ee03-0e52-4097-a37a-df3b968c998e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:25.957326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875ee03-0e52-4097-a37a-df3b968c998e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:25.991843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82dfff-1b36-4a06-ab4c-bd665e442f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:25.992475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82dfff-1b36-4a06-ab4c-bd665e442f96 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:25.992764 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82dfff-1b36-4a06-ab4c-bd665e442f96 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:25.993043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82dfff-1b36-4a06-ab4c-bd665e442f96 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:25.993810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82dfff-1b36-4a06-ab4c-bd665e442f96 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:25.994120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82dfff-1b36-4a06-ab4c-bd665e442f96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:25.995508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82dfff-1b36-4a06-ab4c-bd665e442f96 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:26.004961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 65/141] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:26.013499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d204bc-4e45-4055-9c59-7bb980a09af2 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:26.087091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:26.158752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d204bc-4e45-4055-9c59-7bb980a09af2 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:26.202369 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa883461-d737-461d-a1ba-ed67cdc10abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:26.203284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883461-d737-461d-a1ba-ed67cdc10abe None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:26.206140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883461-d737-461d-a1ba-ed67cdc10abe None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:26.206431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883461-d737-461d-a1ba-ed67cdc10abe None admin] PATH_INFO: `/v3/users/10af1686119142b1a26f7b138e6faaeb` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:26.207977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa883461-d737-461d-a1ba-ed67cdc10abe None admin] RBAC: Authorizing `identity:get_user(user_id=10af1686119142b1a26f7b138e6faaeb)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:26.209499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa883461-d737-461d-a1ba-ed67cdc10abe None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:26.211433 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 77/142] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:26 2022] GET /identity/v3/users/10af1686119142b1a26f7b138e6faaeb => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:26.217226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82dfff-1b36-4a06-ab4c-bd665e442f96 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:26.248527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:26.348877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82dfff-1b36-4a06-ab4c-bd665e442f96 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:26.384389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b79be-752b-4c5d-9ea4-4dda6539e3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:26.385269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b79be-752b-4c5d-9ea4-4dda6539e3be None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:26.385558 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b79be-752b-4c5d-9ea4-4dda6539e3be None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:26.385839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b79be-752b-4c5d-9ea4-4dda6539e3be None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:26.387054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b79be-752b-4c5d-9ea4-4dda6539e3be None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:26.388568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b79be-752b-4c5d-9ea4-4dda6539e3be None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:26.390494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 66/143] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:26 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:26.397682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa883461-d737-461d-a1ba-ed67cdc10abe None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:26.546898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa883461-d737-461d-a1ba-ed67cdc10abe None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:26.580542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:26.598453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6c09b-2abd-4a0f-a1cd-1838b57a9496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:26.599354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6c09b-2abd-4a0f-a1cd-1838b57a9496 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:26.599636 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6c09b-2abd-4a0f-a1cd-1838b57a9496 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:26.599910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6c09b-2abd-4a0f-a1cd-1838b57a9496 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008/users/10af1686119142b1a26f7b138e6faaeb/roles/2a5bf966af5a486fa21c10760ebfbcad` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:26.606335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6c09b-2abd-4a0f-a1cd-1838b57a9496 None admin] RBAC: Authorizing `identity:create_grant(project_id=9811a47b7d5449f999b0f971ee26f008, user_id=10af1686119142b1a26f7b138e6faaeb, role_id=2a5bf966af5a486fa21c10760ebfbcad)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:26.607879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6c09b-2abd-4a0f-a1cd-1838b57a9496 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:26.614201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a98352-3f22-4088-a36f-03f8a6c4aef2 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:26.614491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a98352-3f22-4088-a36f-03f8a6c4aef2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:26.614789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a98352-3f22-4088-a36f-03f8a6c4aef2 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:26.616512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 67/144] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:26.621960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6aa344-a1b0-40dc-9b6e-871034de0027 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:26.622254 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6aa344-a1b0-40dc-9b6e-871034de0027 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:26.622577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6aa344-a1b0-40dc-9b6e-871034de0027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:26.653396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-7c6aa344-a1b0-40dc-9b6e-871034de0027 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:26.655307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7c6aa344-a1b0-40dc-9b6e-871034de0027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:26.698840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 78/145] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:36:26 2022] PUT /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008/users/10af1686119142b1a26f7b138e6faaeb/roles/2a5bf966af5a486fa21c10760ebfbcad => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:26.728062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 68/146] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:26.751505 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6c09b-2abd-4a0f-a1cd-1838b57a9496 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:26.878753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c55dd-cf7b-446d-b3d4-a3b1bc479786 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:26.879061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c55dd-cf7b-446d-b3d4-a3b1bc479786 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:26.879236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c55dd-cf7b-446d-b3d4-a3b1bc479786 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:26.880846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 69/147] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:26.887013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6c09b-2abd-4a0f-a1cd-1838b57a9496 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:26.897490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138272-7fe9-4eb5-b639-4fc8909328c0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:26.897792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138272-7fe9-4eb5-b639-4fc8909328c0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:26.898061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138272-7fe9-4eb5-b639-4fc8909328c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:26.923044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-fc138272-7fe9-4eb5-b639-4fc8909328c0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:26.924269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-fc138272-7fe9-4eb5-b639-4fc8909328c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:26.952799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:26.983806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d48d3d-594e-4b47-b9bf-0f157a9bbed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:26.985010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d48d3d-594e-4b47-b9bf-0f157a9bbed9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:26.985492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d48d3d-594e-4b47-b9bf-0f157a9bbed9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:26.985983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d48d3d-594e-4b47-b9bf-0f157a9bbed9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:27.016599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d48d3d-594e-4b47-b9bf-0f157a9bbed9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:27.023738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 70/148] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-a2d48d3d-594e-4b47-b9bf-0f157a9bbed9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:27.024931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:27.025771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:27.026554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:27.029218 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 79/149] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:36:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:27.035029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc138272-7fe9-4eb5-b639-4fc8909328c0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:27.071365 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d48d3d-594e-4b47-b9bf-0f157a9bbed9 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:27.127300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:27.164225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc138272-7fe9-4eb5-b639-4fc8909328c0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:27.194037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d48d3d-594e-4b47-b9bf-0f157a9bbed9 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:27.212706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf0fcf-ccf0-40e3-8015-0f15d234d500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:27.214520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf0fcf-ccf0-40e3-8015-0f15d234d500 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:27.215771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf0fcf-ccf0-40e3-8015-0f15d234d500 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:27.216283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf0fcf-ccf0-40e3-8015-0f15d234d500 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:27.217441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf0fcf-ccf0-40e3-8015-0f15d234d500 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:27.224375 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf0fcf-ccf0-40e3-8015-0f15d234d500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:27.224375 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf0fcf-ccf0-40e3-8015-0f15d234d500 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:27.236237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 71/150] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:27.245691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf0fcf-ccf0-40e3-8015-0f15d234d500 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:27.266222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:27.266439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80ba39-fb72-4f4e-8ff9-cafdc7c7b26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:27.267425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80ba39-fb72-4f4e-8ff9-cafdc7c7b26f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:27.267851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80ba39-fb72-4f4e-8ff9-cafdc7c7b26f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:27.272736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80ba39-fb72-4f4e-8ff9-cafdc7c7b26f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:27.275354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80ba39-fb72-4f4e-8ff9-cafdc7c7b26f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:27.277011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80ba39-fb72-4f4e-8ff9-cafdc7c7b26f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:27.278863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 80/151] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:27.285472 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80ba39-fb72-4f4e-8ff9-cafdc7c7b26f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:27.467089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf0fcf-ccf0-40e3-8015-0f15d234d500 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:27.522554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80ba39-fb72-4f4e-8ff9-cafdc7c7b26f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:27.529436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:27.562244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0737c4-1731-42b4-a002-a17d42085833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:27.563375 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0737c4-1731-42b4-a002-a17d42085833 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:27.563832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0737c4-1731-42b4-a002-a17d42085833 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:27.564258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0737c4-1731-42b4-a002-a17d42085833 None admin] PATH_INFO: `/v3/users/0840c2e542b841edbc74432280137f72` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:27.569101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0737c4-1731-42b4-a002-a17d42085833 None admin] RBAC: Authorizing `identity:get_user(user_id=0840c2e542b841edbc74432280137f72)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:27.570759 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0737c4-1731-42b4-a002-a17d42085833 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:27.572593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 72/152] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:27 2022] GET /identity/v3/users/0840c2e542b841edbc74432280137f72 => generated 247 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:27.581467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0737c4-1731-42b4-a002-a17d42085833 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:27.596406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42703dfd-5447-4d0e-8eda-cd1f48cf6958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:27.596406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42703dfd-5447-4d0e-8eda-cd1f48cf6958 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:27.596406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42703dfd-5447-4d0e-8eda-cd1f48cf6958 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:27.596406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42703dfd-5447-4d0e-8eda-cd1f48cf6958 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:27.596833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42703dfd-5447-4d0e-8eda-cd1f48cf6958 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:27.597204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42703dfd-5447-4d0e-8eda-cd1f48cf6958 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:27.684706 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 81/153] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:36:27 2022] POST /identity/v3/users => generated 283 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:27.733312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:27.757916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0737c4-1731-42b4-a002-a17d42085833 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:27.796408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e7920-483f-401b-872f-e29c617224e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:27.797308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e7920-483f-401b-872f-e29c617224e5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:27.797744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e7920-483f-401b-872f-e29c617224e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:27.798190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e7920-483f-401b-872f-e29c617224e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:27.884482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e7920-483f-401b-872f-e29c617224e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:27.886236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e7920-483f-401b-872f-e29c617224e5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:27.994450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:28.028310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 73/154] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:36:28.034632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42703dfd-5447-4d0e-8eda-cd1f48cf6958 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:28.189690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42703dfd-5447-4d0e-8eda-cd1f48cf6958 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:28.237176 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c9199-7b17-4845-8b6e-d63e3f7bea75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:28.238302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c9199-7b17-4845-8b6e-d63e3f7bea75 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:28.238733 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c9199-7b17-4845-8b6e-d63e3f7bea75 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:28.239157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c9199-7b17-4845-8b6e-d63e3f7bea75 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:28.245267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c9199-7b17-4845-8b6e-d63e3f7bea75 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:28.246957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c9199-7b17-4845-8b6e-d63e3f7bea75 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:28.249074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 82/155] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:28 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:28.254961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e7920-483f-401b-872f-e29c617224e5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:28.395792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e7920-483f-401b-872f-e29c617224e5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:28.441972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab0aff-dd9c-4959-8be1-ce91aecc72d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:28.442916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab0aff-dd9c-4959-8be1-ce91aecc72d5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:28.443349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab0aff-dd9c-4959-8be1-ce91aecc72d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:28.443776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab0aff-dd9c-4959-8be1-ce91aecc72d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:28.444797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab0aff-dd9c-4959-8be1-ce91aecc72d5 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=9811a47b7d5449f999b0f971ee26f008, user.id=0840c2e542b841edbc74432280137f72) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:28.445274 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab0aff-dd9c-4959-8be1-ce91aecc72d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:28.446873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab0aff-dd9c-4959-8be1-ce91aecc72d5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:28.457007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 74/156] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:28 2022] GET /identity/v3/role_assignments?user.id=0840c2e542b841edbc74432280137f72&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=04c11dea81274a69974ad5c639f2c60b => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:28.483024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e524ce-c6e6-436f-8c5c-8126c8a43ee8 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:28.483545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e524ce-c6e6-436f-8c5c-8126c8a43ee8 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:28.483972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e524ce-c6e6-436f-8c5c-8126c8a43ee8 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:28.486020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 83/157] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:28.491481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fedd-7853-4347-881a-089197314da2 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:28.491999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fedd-7853-4347-881a-089197314da2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:28.492455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fedd-7853-4347-881a-089197314da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:28.517767 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-49f3fedd-7853-4347-881a-089197314da2 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:28.520527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e38f7b-0dfe-46d5-9472-33e46c2fe244 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:28.520527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e38f7b-0dfe-46d5-9472-33e46c2fe244 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:28.520527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e38f7b-0dfe-46d5-9472-33e46c2fe244 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:28.520803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-49f3fedd-7853-4347-881a-089197314da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:28.521208 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 84/158] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:28.527302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b440b-b29d-4d5f-80fd-b95b6b3f169a None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:28.529278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b440b-b29d-4d5f-80fd-b95b6b3f169a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:28.529999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b440b-b29d-4d5f-80fd-b95b6b3f169a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:28.560327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2e7b440b-b29d-4d5f-80fd-b95b6b3f169a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:28.561618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2e7b440b-b29d-4d5f-80fd-b95b6b3f169a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:28.593079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:28.617365 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:28.637704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 75/159] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:28.639441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d6c8a-2491-4dae-893b-dac075f5d6ae None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:28.639970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d6c8a-2491-4dae-893b-dac075f5d6ae None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:28.640472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d6c8a-2491-4dae-893b-dac075f5d6ae None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:28.642009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 76/160] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:28.651169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fbf21-a837-4cf5-817b-0428de1fc476 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:28.651780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fbf21-a837-4cf5-817b-0428de1fc476 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:28.652255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fbf21-a837-4cf5-817b-0428de1fc476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:28.677372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b36fbf21-a837-4cf5-817b-0428de1fc476 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:28.678915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b36fbf21-a837-4cf5-817b-0428de1fc476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:28.701108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 85/161] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:28.704399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b440b-b29d-4d5f-80fd-b95b6b3f169a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:28.789293 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 77/162] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:28.790162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fbf21-a837-4cf5-817b-0428de1fc476 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:28.863036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b440b-b29d-4d5f-80fd-b95b6b3f169a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:28.926082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:28.926432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0dfda-5ae2-48c8-8fcb-00b33dd4c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:28.927821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0dfda-5ae2-48c8-8fcb-00b33dd4c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:28.927821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0dfda-5ae2-48c8-8fcb-00b33dd4c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:28.927982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0dfda-5ae2-48c8-8fcb-00b33dd4c2d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:28.928929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fbf21-a837-4cf5-817b-0428de1fc476 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:28.953854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0dfda-5ae2-48c8-8fcb-00b33dd4c2d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-71b0dfda-5ae2-48c8-8fcb-00b33dd4c2d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:28.963776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:28.964854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:28.965628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:28.968873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 86/163] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:28.969985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0dfda-5ae2-48c8-8fcb-00b33dd4c2d9 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:28.979698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26595f77-aab9-4bf6-b914-895b7fad10df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:28.980570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595f77-aab9-4bf6-b914-895b7fad10df None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:28.980861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595f77-aab9-4bf6-b914-895b7fad10df None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:28.981143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26595f77-aab9-4bf6-b914-895b7fad10df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:28.998034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26595f77-aab9-4bf6-b914-895b7fad10df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:28.998755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-26595f77-aab9-4bf6-b914-895b7fad10df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:29.007964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:29.015121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 78/164] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:36:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:29.015904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26595f77-aab9-4bf6-b914-895b7fad10df None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:29.105709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0dfda-5ae2-48c8-8fcb-00b33dd4c2d9 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:29.147401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa536b-4312-492f-84b8-8aa3cad22332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:29.148554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa536b-4312-492f-84b8-8aa3cad22332 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:29.148845 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa536b-4312-492f-84b8-8aa3cad22332 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:29.149131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa536b-4312-492f-84b8-8aa3cad22332 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:29.164684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26595f77-aab9-4bf6-b914-895b7fad10df None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:29.172247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa536b-4312-492f-84b8-8aa3cad22332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-39aa536b-4312-492f-84b8-8aa3cad22332 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:29.181575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:29.182769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:29.183532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 87/165] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:29.184180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa536b-4312-492f-84b8-8aa3cad22332 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:29.228711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344d541-b7e4-427e-99b0-401526a0c74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:29.229405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344d541-b7e4-427e-99b0-401526a0c74d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:29.229692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344d541-b7e4-427e-99b0-401526a0c74d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:29.229970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344d541-b7e4-427e-99b0-401526a0c74d None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:29.230739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344d541-b7e4-427e-99b0-401526a0c74d None admin] RBAC: Adding query filter params (name=member) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:29.231052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344d541-b7e4-427e-99b0-401526a0c74d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:29.232536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344d541-b7e4-427e-99b0-401526a0c74d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:29.250383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 79/166] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:29.251383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344d541-b7e4-427e-99b0-401526a0c74d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:29.296571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:29.333375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:29.367896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa536b-4312-492f-84b8-8aa3cad22332 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:29.430116 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344d541-b7e4-427e-99b0-401526a0c74d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:29.432749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15517d3e-7949-45d3-a48d-97c8327b2dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:29.433437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15517d3e-7949-45d3-a48d-97c8327b2dff None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:29.433724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15517d3e-7949-45d3-a48d-97c8327b2dff None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:29.434006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15517d3e-7949-45d3-a48d-97c8327b2dff None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:29.434778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15517d3e-7949-45d3-a48d-97c8327b2dff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:29.435630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15517d3e-7949-45d3-a48d-97c8327b2dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:29.438072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15517d3e-7949-45d3-a48d-97c8327b2dff None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:29.455047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 88/167] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:36:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:29.455909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15517d3e-7949-45d3-a48d-97c8327b2dff None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:29.501900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226af71-2b03-4c4b-8af8-12d775b07ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:29.502579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226af71-2b03-4c4b-8af8-12d775b07ebf None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:29.502862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226af71-2b03-4c4b-8af8-12d775b07ebf None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:29.503146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226af71-2b03-4c4b-8af8-12d775b07ebf None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:29.505893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226af71-2b03-4c4b-8af8-12d775b07ebf None admin] RBAC: Adding query filter params (name=member) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:29.506206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226af71-2b03-4c4b-8af8-12d775b07ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:29.507587 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226af71-2b03-4c4b-8af8-12d775b07ebf None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:29.527401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 80/168] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:29.528247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226af71-2b03-4c4b-8af8-12d775b07ebf None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:29.610145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15517d3e-7949-45d3-a48d-97c8327b2dff None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:29.679494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02471476-62bf-4b4d-80f5-620f74ba14eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:29.680502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02471476-62bf-4b4d-80f5-620f74ba14eb None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:29.680787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02471476-62bf-4b4d-80f5-620f74ba14eb None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:29.681062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02471476-62bf-4b4d-80f5-620f74ba14eb None admin] PATH_INFO: `/v3/users/7384fa3396274b4e8aa20b57838b11a6` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:29.683000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02471476-62bf-4b4d-80f5-620f74ba14eb None admin] RBAC: Authorizing `identity:get_user(user_id=7384fa3396274b4e8aa20b57838b11a6)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:29.684481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02471476-62bf-4b4d-80f5-620f74ba14eb None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:29.689071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 89/169] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:29 2022] GET /identity/v3/users/7384fa3396274b4e8aa20b57838b11a6 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:29.692124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02471476-62bf-4b4d-80f5-620f74ba14eb None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:29.724390 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:29.732927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:29.763017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226af71-2b03-4c4b-8af8-12d775b07ebf None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:29.824302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd241db8-7c08-44c0-b390-6534809c6c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:29.825210 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd241db8-7c08-44c0-b390-6534809c6c0c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:29.825497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd241db8-7c08-44c0-b390-6534809c6c0c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:29.826891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd241db8-7c08-44c0-b390-6534809c6c0c None admin] PATH_INFO: `/v3/users/7384fa3396274b4e8aa20b57838b11a6` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:29.830819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd241db8-7c08-44c0-b390-6534809c6c0c None admin] RBAC: Authorizing `identity:get_user(user_id=7384fa3396274b4e8aa20b57838b11a6)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:29.832429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd241db8-7c08-44c0-b390-6534809c6c0c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:29.834419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 81/170] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:29 2022] GET /identity/v3/users/7384fa3396274b4e8aa20b57838b11a6 => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:29.835200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd241db8-7c08-44c0-b390-6534809c6c0c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:29.890777 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02471476-62bf-4b4d-80f5-620f74ba14eb None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:29.930231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b00ad-83c6-45d6-a84a-7ef91873c7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:29.931319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b00ad-83c6-45d6-a84a-7ef91873c7e7 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:29.931612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b00ad-83c6-45d6-a84a-7ef91873c7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:29.931913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b00ad-83c6-45d6-a84a-7ef91873c7e7 None admin] PATH_INFO: `/v3/users/7384fa3396274b4e8aa20b57838b11a6` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:29.933254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b00ad-83c6-45d6-a84a-7ef91873c7e7 None admin] RBAC: Authorizing `identity:get_user(user_id=7384fa3396274b4e8aa20b57838b11a6)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:29.935550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b00ad-83c6-45d6-a84a-7ef91873c7e7 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:29.938335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 90/171] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:29 2022] GET /identity/v3/users/7384fa3396274b4e8aa20b57838b11a6 => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:29.939207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b00ad-83c6-45d6-a84a-7ef91873c7e7 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:30.033247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:30.056529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b00ad-83c6-45d6-a84a-7ef91873c7e7 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:30.094999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd241db8-7c08-44c0-b390-6534809c6c0c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:30.097722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e20c4-f15a-462d-93d6-81793c4ec0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:30.098388 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e20c4-f15a-462d-93d6-81793c4ec0df None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:30.098679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e20c4-f15a-462d-93d6-81793c4ec0df None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:30.098960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e20c4-f15a-462d-93d6-81793c4ec0df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:30.178074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e20c4-f15a-462d-93d6-81793c4ec0df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:30.179709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e20c4-f15a-462d-93d6-81793c4ec0df None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:30.209761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d51d6d-7521-403c-bd5b-58a4c417e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:30.210360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d51d6d-7521-403c-bd5b-58a4c417e66d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:30.210641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d51d6d-7521-403c-bd5b-58a4c417e66d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:30.210918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d51d6d-7521-403c-bd5b-58a4c417e66d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:30.243879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:30.300720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:30.346673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 91/172] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:30.350046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d51d6d-7521-403c-bd5b-58a4c417e66d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:30.352177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d51d6d-7521-403c-bd5b-58a4c417e66d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:30.354285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e20c4-f15a-462d-93d6-81793c4ec0df None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:30.546253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e20c4-f15a-462d-93d6-81793c4ec0df None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:30.550846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 82/173] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:30.552216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e1b39-686b-43c0-999e-cb71f11c7982 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:30.552537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e1b39-686b-43c0-999e-cb71f11c7982 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:30.552818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e1b39-686b-43c0-999e-cb71f11c7982 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:30.554097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 83/174] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:30.554799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e1b39-686b-43c0-999e-cb71f11c7982 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:30.595218 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d014e-f288-45cc-8714-1753bab1c6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:30.595884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d014e-f288-45cc-8714-1753bab1c6f8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:30.596173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d014e-f288-45cc-8714-1753bab1c6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:30.596484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d014e-f288-45cc-8714-1753bab1c6f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:30.648869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:30.674131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d014e-f288-45cc-8714-1753bab1c6f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:30.675619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d014e-f288-45cc-8714-1753bab1c6f8 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:30.680713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:30.785071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 92/175] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:30.785941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d014e-f288-45cc-8714-1753bab1c6f8 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:30.795817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e1b39-686b-43c0-999e-cb71f11c7982 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:30.865131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac436d-13f5-4923-83d2-153252211d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:30.865928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac436d-13f5-4923-83d2-153252211d39 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:30.866213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac436d-13f5-4923-83d2-153252211d39 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:30.866487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac436d-13f5-4923-83d2-153252211d39 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:30.867370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:30.867707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac436d-13f5-4923-83d2-153252211d39 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:30.869177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac436d-13f5-4923-83d2-153252211d39 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:30.870885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 84/176] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:30 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:30.872035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044e46a-5b8c-4da1-bd9f-3ebbfac771e7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:30.872320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044e46a-5b8c-4da1-bd9f-3ebbfac771e7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:30.873157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044e46a-5b8c-4da1-bd9f-3ebbfac771e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:30.887696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d014e-f288-45cc-8714-1753bab1c6f8 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:30.899118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8044e46a-5b8c-4da1-bd9f-3ebbfac771e7 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:30.900320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8044e46a-5b8c-4da1-bd9f-3ebbfac771e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:30.922560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef819bd-40a9-4005-b402-1085f577c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:30.923482 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef819bd-40a9-4005-b402-1085f577c3af None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:30.923767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef819bd-40a9-4005-b402-1085f577c3af None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:30.924044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef819bd-40a9-4005-b402-1085f577c3af None admin] PATH_INFO: `/v3/projects/da738cb1cdc344cb8f742cc1f27f070c` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:30.925300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef819bd-40a9-4005-b402-1085f577c3af None admin] RBAC: Authorizing `identity:get_project(project_id=da738cb1cdc344cb8f742cc1f27f070c)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:30.926751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef819bd-40a9-4005-b402-1085f577c3af None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:30.928865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 93/177] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:30 2022] GET /identity/v3/projects/da738cb1cdc344cb8f742cc1f27f070c => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:30.934559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef819bd-40a9-4005-b402-1085f577c3af None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:30.982669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 85/178] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:30.984122 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044e46a-5b8c-4da1-bd9f-3ebbfac771e7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:31.040806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:31.101703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044e46a-5b8c-4da1-bd9f-3ebbfac771e7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:31.153156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e060d63-4251-4e63-bc47-be8ce9800aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:31.153705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e060d63-4251-4e63-bc47-be8ce9800aa0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:31.154019 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e060d63-4251-4e63-bc47-be8ce9800aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:31.154295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e060d63-4251-4e63-bc47-be8ce9800aa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:31.155136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e060d63-4251-4e63-bc47-be8ce9800aa0 None admin] RBAC: Adding query filter params (role.id=f7e0a506c7894b20b2b65fedfd2244e5, scope.project.id=9811a47b7d5449f999b0f971ee26f008, user.id=7384fa3396274b4e8aa20b57838b11a6) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:31.155455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e060d63-4251-4e63-bc47-be8ce9800aa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:31.156986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e060d63-4251-4e63-bc47-be8ce9800aa0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:31.173434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef819bd-40a9-4005-b402-1085f577c3af None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:31.178814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 86/179] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:30 2022] GET /identity/v3/role_assignments?user.id=7384fa3396274b4e8aa20b57838b11a6&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=f7e0a506c7894b20b2b65fedfd2244e5 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:31.179679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e060d63-4251-4e63-bc47-be8ce9800aa0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:31.214323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb2f2b-c424-4a43-bb96-d9b80b66ab02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:31.215745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb2f2b-c424-4a43-bb96-d9b80b66ab02 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:31.216259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb2f2b-c424-4a43-bb96-d9b80b66ab02 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:31.216802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb2f2b-c424-4a43-bb96-d9b80b66ab02 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:31.218318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb2f2b-c424-4a43-bb96-d9b80b66ab02 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:31.220660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb2f2b-c424-4a43-bb96-d9b80b66ab02 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:31.222769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 94/180] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:30 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:31.223766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb2f2b-c424-4a43-bb96-d9b80b66ab02 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:31.248651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:31.341953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb2f2b-c424-4a43-bb96-d9b80b66ab02 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:31.385139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcc54d-683c-42a8-b584-515b60d1fff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:31.386797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcc54d-683c-42a8-b584-515b60d1fff8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:31.387659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcc54d-683c-42a8-b584-515b60d1fff8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:31.389105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcc54d-683c-42a8-b584-515b60d1fff8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:31.413315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bcc54d-683c-42a8-b584-515b60d1fff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-59bcc54d-683c-42a8-b584-515b60d1fff8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:31.427750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:31.428760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:31.429546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:31.433712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e060d63-4251-4e63-bc47-be8ce9800aa0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:31.438387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 95/181] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:31.439518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcc54d-683c-42a8-b584-515b60d1fff8 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:31.455476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:31.486247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b74cc0-95b8-48d4-adf6-4757d1bd466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:31.487690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b74cc0-95b8-48d4-adf6-4757d1bd466c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:31.488324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b74cc0-95b8-48d4-adf6-4757d1bd466c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:31.488981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b74cc0-95b8-48d4-adf6-4757d1bd466c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:31.490173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b74cc0-95b8-48d4-adf6-4757d1bd466c None admin] RBAC: Adding query filter params (role.id=177939861d6e4e82b48401be184d66b4, scope.project.id=da738cb1cdc344cb8f742cc1f27f070c, user.id=7384fa3396274b4e8aa20b57838b11a6) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:31.490966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b74cc0-95b8-48d4-adf6-4757d1bd466c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:31.492943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b74cc0-95b8-48d4-adf6-4757d1bd466c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:31.500779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 87/182] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:31 2022] GET /identity/v3/role_assignments?user.id=7384fa3396274b4e8aa20b57838b11a6&scope.project.id=da738cb1cdc344cb8f742cc1f27f070c&role.id=177939861d6e4e82b48401be184d66b4 => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:31.502348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3765f8-11d5-418f-adcf-46f98db79f1e None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:31.502817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3765f8-11d5-418f-adcf-46f98db79f1e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:31.503798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3765f8-11d5-418f-adcf-46f98db79f1e None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:31.505838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 88/183] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:31.506850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3765f8-11d5-418f-adcf-46f98db79f1e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:31.549196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:31.586952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcc54d-683c-42a8-b584-515b60d1fff8 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:31.631442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15afc7-c9e1-403e-aac8-83118bf33ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:31.632297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15afc7-c9e1-403e-aac8-83118bf33ee5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:31.632697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15afc7-c9e1-403e-aac8-83118bf33ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:31.633039 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15afc7-c9e1-403e-aac8-83118bf33ee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:31.634313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15afc7-c9e1-403e-aac8-83118bf33ee5 None admin] RBAC: Adding query filter params (role.id=177939861d6e4e82b48401be184d66b4, scope.project.id=9811a47b7d5449f999b0f971ee26f008, user.id=7384fa3396274b4e8aa20b57838b11a6) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:31.634694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15afc7-c9e1-403e-aac8-83118bf33ee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:31.636251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15afc7-c9e1-403e-aac8-83118bf33ee5 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:31.643480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 96/184] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:31 2022] GET /identity/v3/role_assignments?user.id=7384fa3396274b4e8aa20b57838b11a6&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=177939861d6e4e82b48401be184d66b4 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:31.645012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ff982-d240-4b55-b5cc-3023e410e011 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:31.645370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ff982-d240-4b55-b5cc-3023e410e011 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:31.645700 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3765f8-11d5-418f-adcf-46f98db79f1e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:31.645831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ff982-d240-4b55-b5cc-3023e410e011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:31.666997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-f34ff982-d240-4b55-b5cc-3023e410e011 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:31.668597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f34ff982-d240-4b55-b5cc-3023e410e011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:31.697770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71733781-e132-481a-a27a-0ecba2127507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:31.699042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71733781-e132-481a-a27a-0ecba2127507 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:31.699538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71733781-e132-481a-a27a-0ecba2127507 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:31.700138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71733781-e132-481a-a27a-0ecba2127507 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:31.701605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71733781-e132-481a-a27a-0ecba2127507 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:31.702329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71733781-e132-481a-a27a-0ecba2127507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:31.704606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71733781-e132-481a-a27a-0ecba2127507 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:31.717632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 89/185] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:31.728575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71733781-e132-481a-a27a-0ecba2127507 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:31.777596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 97/186] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:31.806802 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:31.822323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ff982-d240-4b55-b5cc-3023e410e011 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:31.929774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71733781-e132-481a-a27a-0ecba2127507 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:31.948724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:31.994025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cedd5-dc77-4de0-bfca-944f90bf78b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:31.995106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cedd5-dc77-4de0-bfca-944f90bf78b3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:31.995395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cedd5-dc77-4de0-bfca-944f90bf78b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:31.995671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cedd5-dc77-4de0-bfca-944f90bf78b3 None admin] PATH_INFO: `/v3/users/10af1686119142b1a26f7b138e6faaeb` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:31.996930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cedd5-dc77-4de0-bfca-944f90bf78b3 None admin] RBAC: Authorizing `identity:get_user(user_id=10af1686119142b1a26f7b138e6faaeb)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:31.998628 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cedd5-dc77-4de0-bfca-944f90bf78b3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:32.000584 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 90/187] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:31 2022] GET /identity/v3/users/10af1686119142b1a26f7b138e6faaeb => generated 289 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:32.001803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3c66d-3ee4-4eee-939c-a942bbd363c4 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.002087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3c66d-3ee4-4eee-939c-a942bbd363c4 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.002361 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3c66d-3ee4-4eee-939c-a942bbd363c4 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.003726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 91/188] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:32.005288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ff982-d240-4b55-b5cc-3023e410e011 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:32.009487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629f73c-7bf2-4557-ab51-d763de2aa190 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.009774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629f73c-7bf2-4557-ab51-d763de2aa190 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.010053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629f73c-7bf2-4557-ab51-d763de2aa190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.030043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8629f73c-7bf2-4557-ab51-d763de2aa190 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:32.031219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8629f73c-7bf2-4557-ab51-d763de2aa190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:32.042549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838a4a8-dd6e-4c87-b862-a9c1e5fcfe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:32.043478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a4a8-dd6e-4c87-b862-a9c1e5fcfe1a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.043763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a4a8-dd6e-4c87-b862-a9c1e5fcfe1a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.044034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a4a8-dd6e-4c87-b862-a9c1e5fcfe1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.055949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838a4a8-dd6e-4c87-b862-a9c1e5fcfe1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-2838a4a8-dd6e-4c87-b862-a9c1e5fcfe1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:32.064629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:32.065524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:32.066301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 98/189] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:36:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:32.066981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838a4a8-dd6e-4c87-b862-a9c1e5fcfe1a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:32.110873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 92/190] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:32.116989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8aecb-710e-4d7b-8fe3-9a6d1b05c630 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.116989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8aecb-710e-4d7b-8fe3-9a6d1b05c630 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.116989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8aecb-710e-4d7b-8fe3-9a6d1b05c630 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.116989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 93/191] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:32.116989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8aecb-710e-4d7b-8fe3-9a6d1b05c630 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:32.160395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:32.179859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838a4a8-dd6e-4c87-b862-a9c1e5fcfe1a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:32.205444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:32.213810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f266e34-d77c-4026-8173-c9cb88dfa28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:32.214459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266e34-d77c-4026-8173-c9cb88dfa28a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.214742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266e34-d77c-4026-8173-c9cb88dfa28a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.215016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266e34-d77c-4026-8173-c9cb88dfa28a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.219976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8aecb-710e-4d7b-8fe3-9a6d1b05c630 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:32.254658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d879c1-d191-409a-8092-4bc7f3bd1e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:32.255321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d879c1-d191-409a-8092-4bc7f3bd1e8c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.255604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d879c1-d191-409a-8092-4bc7f3bd1e8c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.255882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d879c1-d191-409a-8092-4bc7f3bd1e8c None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.256665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d879c1-d191-409a-8092-4bc7f3bd1e8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:32.256969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d879c1-d191-409a-8092-4bc7f3bd1e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:32.258318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d879c1-d191-409a-8092-4bc7f3bd1e8c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:32.266997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 94/192] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:32.266997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea922d2-dce9-4e2e-a9b5-1842b26c4753 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.266997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea922d2-dce9-4e2e-a9b5-1842b26c4753 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.266997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea922d2-dce9-4e2e-a9b5-1842b26c4753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.286907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-bea922d2-dce9-4e2e-a9b5-1842b26c4753 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:32.287182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f266e34-d77c-4026-8173-c9cb88dfa28a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:32.288096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-bea922d2-dce9-4e2e-a9b5-1842b26c4753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:32.288679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f266e34-d77c-4026-8173-c9cb88dfa28a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:32.352926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 95/193] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:32.353827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea922d2-dce9-4e2e-a9b5-1842b26c4753 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:32.384560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:32.393586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 99/194] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:32.394475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f266e34-d77c-4026-8173-c9cb88dfa28a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:32.422739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:32.452744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea922d2-dce9-4e2e-a9b5-1842b26c4753 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:32.485886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144754f-dafa-4886-825f-b3f0ad8962cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:32.486694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144754f-dafa-4886-825f-b3f0ad8962cf None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.486979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144754f-dafa-4886-825f-b3f0ad8962cf None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.487258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144754f-dafa-4886-825f-b3f0ad8962cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.492380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f266e34-d77c-4026-8173-c9cb88dfa28a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:32.497171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144754f-dafa-4886-825f-b3f0ad8962cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-5144754f-dafa-4886-825f-b3f0ad8962cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:32.505245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:32.506120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:32.506882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 96/195] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:36:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:32.507528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144754f-dafa-4886-825f-b3f0ad8962cf None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:32.528809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63861ea5-6257-4197-ab75-c457b7ad7294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:32.530306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63861ea5-6257-4197-ab75-c457b7ad7294 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.530587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63861ea5-6257-4197-ab75-c457b7ad7294 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.530863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63861ea5-6257-4197-ab75-c457b7ad7294 None admin] PATH_INFO: `/v3/users/e34570f1efd949dbb5edbf6f90a51dae` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.544125 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63861ea5-6257-4197-ab75-c457b7ad7294 None admin] RBAC: Authorizing `identity:get_user(user_id=e34570f1efd949dbb5edbf6f90a51dae)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:32.545614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63861ea5-6257-4197-ab75-c457b7ad7294 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:32.547249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 100/196] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:32 2022] GET /identity/v3/users/e34570f1efd949dbb5edbf6f90a51dae => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:32.548006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63861ea5-6257-4197-ab75-c457b7ad7294 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:32.605749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:32.614870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144754f-dafa-4886-825f-b3f0ad8962cf None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:32.629222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:32.645102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63861ea5-6257-4197-ab75-c457b7ad7294 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:32.648017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85cead-efff-4b1a-b508-7b7492c0cde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:32.648835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85cead-efff-4b1a-b508-7b7492c0cde5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.649121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85cead-efff-4b1a-b508-7b7492c0cde5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.649397 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85cead-efff-4b1a-b508-7b7492c0cde5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.650562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85cead-efff-4b1a-b508-7b7492c0cde5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:32.651999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85cead-efff-4b1a-b508-7b7492c0cde5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:32.653622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 97/197] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:32.654987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85cead-efff-4b1a-b508-7b7492c0cde5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:32.679398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4f854-1e90-4c2e-b535-bb9012916d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:32.680260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4f854-1e90-4c2e-b535-bb9012916d4e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.680572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4f854-1e90-4c2e-b535-bb9012916d4e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.680927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4f854-1e90-4c2e-b535-bb9012916d4e None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.682120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4f854-1e90-4c2e-b535-bb9012916d4e None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:32.683571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4f854-1e90-4c2e-b535-bb9012916d4e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:32.685441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 101/198] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:32 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:32.686188 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4f854-1e90-4c2e-b535-bb9012916d4e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:32.754724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85cead-efff-4b1a-b508-7b7492c0cde5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:32.784156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4f854-1e90-4c2e-b535-bb9012916d4e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:32.787549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3537fe8-ed2e-4c70-8588-963c11ca8957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:32.788210 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3537fe8-ed2e-4c70-8588-963c11ca8957 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.788525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3537fe8-ed2e-4c70-8588-963c11ca8957 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.788803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3537fe8-ed2e-4c70-8588-963c11ca8957 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.789554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3537fe8-ed2e-4c70-8588-963c11ca8957 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:32.789861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3537fe8-ed2e-4c70-8588-963c11ca8957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:32.791220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3537fe8-ed2e-4c70-8588-963c11ca8957 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:32.799004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 98/199] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:36:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:32.799754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3537fe8-ed2e-4c70-8588-963c11ca8957 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:32.816749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881f9fd-8a73-436a-a549-44816af378b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:32.817399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881f9fd-8a73-436a-a549-44816af378b1 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.817685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881f9fd-8a73-436a-a549-44816af378b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.817965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881f9fd-8a73-436a-a549-44816af378b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.820599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:32.839620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:32.884638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881f9fd-8a73-436a-a549-44816af378b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:32.886098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881f9fd-8a73-436a-a549-44816af378b1 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:32.896263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3537fe8-ed2e-4c70-8588-963c11ca8957 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:32.929181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb74bc-b791-47d3-b2fd-4f2b1aaf0c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:32.929843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb74bc-b791-47d3-b2fd-4f2b1aaf0c95 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:32.930126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb74bc-b791-47d3-b2fd-4f2b1aaf0c95 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:32.930401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb74bc-b791-47d3-b2fd-4f2b1aaf0c95 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:32.931112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb74bc-b791-47d3-b2fd-4f2b1aaf0c95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:32.932562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb74bc-b791-47d3-b2fd-4f2b1aaf0c95 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:32.981015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 102/200] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:32.981832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881f9fd-8a73-436a-a549-44816af378b1 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:33.028476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 99/201] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:36:32 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:33.028476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb74bc-b791-47d3-b2fd-4f2b1aaf0c95 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:33.034422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:33.089773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881f9fd-8a73-436a-a549-44816af378b1 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:33.119923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:33.130541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aa5a9-b572-4e60-871d-dd0682800797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:33.131480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aa5a9-b572-4e60-871d-dd0682800797 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:33.131763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aa5a9-b572-4e60-871d-dd0682800797 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:33.132039 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6aa5a9-b572-4e60-871d-dd0682800797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:33.133021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6aa5a9-b572-4e60-871d-dd0682800797 None admin] RBAC: Adding query filter params (role.id=2a5bf966af5a486fa21c10760ebfbcad, scope.project.id=9811a47b7d5449f999b0f971ee26f008, user.id=10af1686119142b1a26f7b138e6faaeb) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:33.133823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6aa5a9-b572-4e60-871d-dd0682800797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:33.135303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6aa5a9-b572-4e60-871d-dd0682800797 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:33.143124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 103/202] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:32 2022] GET /identity/v3/role_assignments?user.id=10af1686119142b1a26f7b138e6faaeb&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=2a5bf966af5a486fa21c10760ebfbcad => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:33.143774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aa5a9-b572-4e60-871d-dd0682800797 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:33.155071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb74bc-b791-47d3-b2fd-4f2b1aaf0c95 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:33.216923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1aeaa-d5a4-4287-8549-9573b446140c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:33.218286 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1aeaa-d5a4-4287-8549-9573b446140c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:33.219025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1aeaa-d5a4-4287-8549-9573b446140c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:33.219310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1aeaa-d5a4-4287-8549-9573b446140c None admin] PATH_INFO: `/v3/users/e34570f1efd949dbb5edbf6f90a51dae` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:33.220825 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1aeaa-d5a4-4287-8549-9573b446140c None admin] RBAC: Authorizing `identity:get_user(user_id=e34570f1efd949dbb5edbf6f90a51dae)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:33.222475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1aeaa-d5a4-4287-8549-9573b446140c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:33.224189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 100/203] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:33 2022] GET /identity/v3/users/e34570f1efd949dbb5edbf6f90a51dae => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:33.230912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1aeaa-d5a4-4287-8549-9573b446140c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:33.273432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6aa5a9-b572-4e60-871d-dd0682800797 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:33.306329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:33.327956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be77169-379f-4297-a01b-a9815a6cc0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:33.329061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be77169-379f-4297-a01b-a9815a6cc0ad None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:33.329342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be77169-379f-4297-a01b-a9815a6cc0ad None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:33.329618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be77169-379f-4297-a01b-a9815a6cc0ad None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:33.330844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be77169-379f-4297-a01b-a9815a6cc0ad None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:33.332455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be77169-379f-4297-a01b-a9815a6cc0ad None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:33.336662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 104/204] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:33 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:33.340907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be77169-379f-4297-a01b-a9815a6cc0ad None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:33.369495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1aeaa-d5a4-4287-8549-9573b446140c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:33.412528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcc1ab-b282-4676-9969-cafd3b940688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:33.413409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcc1ab-b282-4676-9969-cafd3b940688 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:33.413702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcc1ab-b282-4676-9969-cafd3b940688 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:33.413983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcc1ab-b282-4676-9969-cafd3b940688 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:33.434473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:33.480746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be77169-379f-4297-a01b-a9815a6cc0ad None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:33.496479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcc1ab-b282-4676-9969-cafd3b940688 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:33.498682 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcc1ab-b282-4676-9969-cafd3b940688 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:33.520908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31d539-3a90-481d-ae67-5762c9104a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:33.522001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31d539-3a90-481d-ae67-5762c9104a9c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:33.522503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31d539-3a90-481d-ae67-5762c9104a9c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:33.522981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31d539-3a90-481d-ae67-5762c9104a9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:33.524122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31d539-3a90-481d-ae67-5762c9104a9c None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=682d8892f2d3403f862be249dc3a8acf, user.id=e34570f1efd949dbb5edbf6f90a51dae) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:33.524773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31d539-3a90-481d-ae67-5762c9104a9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:33.526455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31d539-3a90-481d-ae67-5762c9104a9c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:33.533134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 105/205] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:33 2022] GET /identity/v3/role_assignments?user.id=e34570f1efd949dbb5edbf6f90a51dae&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=04c11dea81274a69974ad5c639f2c60b => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:33.617604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 101/206] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:36:33.625522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31d539-3a90-481d-ae67-5762c9104a9c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:33.696767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:33.738798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31d539-3a90-481d-ae67-5762c9104a9c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:33.775599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557661f-63b2-49c4-8efa-6ce93b7dce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:33.776495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557661f-63b2-49c4-8efa-6ce93b7dce44 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:33.776776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557661f-63b2-49c4-8efa-6ce93b7dce44 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:33.777054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557661f-63b2-49c4-8efa-6ce93b7dce44 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:33.778291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557661f-63b2-49c4-8efa-6ce93b7dce44 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:33.780301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557661f-63b2-49c4-8efa-6ce93b7dce44 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:33.786817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 106/207] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:33 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:33.788255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcc1ab-b282-4676-9969-cafd3b940688 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:33.835087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:33.893268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcc1ab-b282-4676-9969-cafd3b940688 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:33.928478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f3cfd-acef-4b90-97c9-e508ef3421ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:33.929510 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f3cfd-acef-4b90-97c9-e508ef3421ef None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:33.930001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f3cfd-acef-4b90-97c9-e508ef3421ef None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:33.930496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f3cfd-acef-4b90-97c9-e508ef3421ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:33.931562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f3cfd-acef-4b90-97c9-e508ef3421ef None admin] RBAC: Adding query filter params (role.id=f7e0a506c7894b20b2b65fedfd2244e5, scope.project.id=682d8892f2d3403f862be249dc3a8acf, user.id=e34570f1efd949dbb5edbf6f90a51dae) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:33.932110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f3cfd-acef-4b90-97c9-e508ef3421ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:33.933796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f3cfd-acef-4b90-97c9-e508ef3421ef None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:33.940031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 102/208] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:33 2022] GET /identity/v3/role_assignments?user.id=e34570f1efd949dbb5edbf6f90a51dae&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=f7e0a506c7894b20b2b65fedfd2244e5 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:35.492581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef374eb-285b-412a-8f43-5da48b1e1061 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:35.493612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef374eb-285b-412a-8f43-5da48b1e1061 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:35.494095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef374eb-285b-412a-8f43-5da48b1e1061 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:35.495671 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 107/209] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:35.498332 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d2dbb-d9a9-43c8-8403-8055648ddf05 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:35.499180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d2dbb-d9a9-43c8-8403-8055648ddf05 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:35.499680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d2dbb-d9a9-43c8-8403-8055648ddf05 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:35.501312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 103/210] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:35.501559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c1008-d509-41a9-8b98-f42cfaccec43 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:35.502056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c1008-d509-41a9-8b98-f42cfaccec43 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:35.502651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c1008-d509-41a9-8b98-f42cfaccec43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:35.506304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134260fc-76d6-4d31-89e1-4138f8f8e2c8 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:35.506814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134260fc-76d6-4d31-89e1-4138f8f8e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:35.507300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134260fc-76d6-4d31-89e1-4138f8f8e2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:35.521315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a96c1008-d509-41a9-8b98-f42cfaccec43 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:35.522500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a96c1008-d509-41a9-8b98-f42cfaccec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:35.525395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-134260fc-76d6-4d31-89e1-4138f8f8e2c8 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:35.526646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-134260fc-76d6-4d31-89e1-4138f8f8e2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:35.607182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 108/211] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:35.619116 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 104/212] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:35.633066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c1008-d509-41a9-8b98-f42cfaccec43 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:35.658524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134260fc-76d6-4d31-89e1-4138f8f8e2c8 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:35.683543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:35.712413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:35.741139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c1008-d509-41a9-8b98-f42cfaccec43 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:35.778549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930a40c-e69d-4b19-bd3d-e5efbfca3ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:35.779748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930a40c-e69d-4b19-bd3d-e5efbfca3ca3 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:35.780677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930a40c-e69d-4b19-bd3d-e5efbfca3ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:35.781146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930a40c-e69d-4b19-bd3d-e5efbfca3ca3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:35.782928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930a40c-e69d-4b19-bd3d-e5efbfca3ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:35.785254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930a40c-e69d-4b19-bd3d-e5efbfca3ca3 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:35.787805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 109/213] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:35.789402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114c949-122b-4148-8d7b-55fb42de2f15 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:35.790192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114c949-122b-4148-8d7b-55fb42de2f15 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:35.790628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114c949-122b-4148-8d7b-55fb42de2f15 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:35.792572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 110/214] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:35.793416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114c949-122b-4148-8d7b-55fb42de2f15 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:35.797168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134260fc-76d6-4d31-89e1-4138f8f8e2c8 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:35.833471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb27b4-87f0-4bb1-8883-e0a48177aab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:35.834268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb27b4-87f0-4bb1-8883-e0a48177aab9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:35.834547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb27b4-87f0-4bb1-8883-e0a48177aab9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:35.834822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb27b4-87f0-4bb1-8883-e0a48177aab9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:35.844984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb27b4-87f0-4bb1-8883-e0a48177aab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-9dfb27b4-87f0-4bb1-8883-e0a48177aab9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:35.852638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:35.853491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:35.854271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 105/215] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:35.855451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b8025-eade-4970-be7a-21eeb7633597 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:35.855451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b8025-eade-4970-be7a-21eeb7633597 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:35.855580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b8025-eade-4970-be7a-21eeb7633597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:35.873367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-458b8025-eade-4970-be7a-21eeb7633597 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:35.874502 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-458b8025-eade-4970-be7a-21eeb7633597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:35.910657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114c949-122b-4148-8d7b-55fb42de2f15 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:35.926327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:35.941887 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 106/216] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:35.942743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b8025-eade-4970-be7a-21eeb7633597 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:35.952684 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aaa0c-0934-43f3-a4b0-8c4eb8b4f50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:35.953924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aaa0c-0934-43f3-a4b0-8c4eb8b4f50e None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:35.954278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aaa0c-0934-43f3-a4b0-8c4eb8b4f50e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:35.954624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aaa0c-0934-43f3-a4b0-8c4eb8b4f50e None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:35.955450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2aaa0c-0934-43f3-a4b0-8c4eb8b4f50e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:35.957095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2aaa0c-0934-43f3-a4b0-8c4eb8b4f50e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:35.983039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:36.046028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 111/217] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:36:35 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:36.046028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aaa0c-0934-43f3-a4b0-8c4eb8b4f50e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:36.065686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b8025-eade-4970-be7a-21eeb7633597 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:36.117340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d303a-eec4-4867-9474-bba9f8cbb7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:36.118069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d303a-eec4-4867-9474-bba9f8cbb7b6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.119028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d303a-eec4-4867-9474-bba9f8cbb7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.119359 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d303a-eec4-4867-9474-bba9f8cbb7b6 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.120141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d303a-eec4-4867-9474-bba9f8cbb7b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:36.120468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d303a-eec4-4867-9474-bba9f8cbb7b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:36.122443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d303a-eec4-4867-9474-bba9f8cbb7b6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:36.136024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 107/218] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:36.144635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d303a-eec4-4867-9474-bba9f8cbb7b6 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:36.180642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aaa0c-0934-43f3-a4b0-8c4eb8b4f50e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:36.220300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f87cc-7d09-43a1-9d29-f6c8899502dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:36.221402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f87cc-7d09-43a1-9d29-f6c8899502dc None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.221832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f87cc-7d09-43a1-9d29-f6c8899502dc None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.222294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f87cc-7d09-43a1-9d29-f6c8899502dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.235774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f87cc-7d09-43a1-9d29-f6c8899502dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-100f87cc-7d09-43a1-9d29-f6c8899502dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:36.243999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:36.244921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:36.245709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 112/219] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:36:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:36.246877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f1d9b-97b7-4595-a9e8-eadb87e3b93f None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.246877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f1d9b-97b7-4595-a9e8-eadb87e3b93f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.247041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f1d9b-97b7-4595-a9e8-eadb87e3b93f None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.248519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 113/220] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:36.252165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:36.253206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f1d9b-97b7-4595-a9e8-eadb87e3b93f None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:36.266610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d303a-eec4-4867-9474-bba9f8cbb7b6 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:36.300757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:36.306227 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15a990-8532-45aa-a5d4-a503e1eda476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:36.307089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15a990-8532-45aa-a5d4-a503e1eda476 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.307370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15a990-8532-45aa-a5d4-a503e1eda476 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.307651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15a990-8532-45aa-a5d4-a503e1eda476 None admin] PATH_INFO: `/v3/users/618682c43f9c4ab29adf65cff57f6aa3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.326598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15a990-8532-45aa-a5d4-a503e1eda476 None admin] RBAC: Authorizing `identity:get_user(user_id=618682c43f9c4ab29adf65cff57f6aa3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:36.328071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15a990-8532-45aa-a5d4-a503e1eda476 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:36.330524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 108/221] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:36 2022] GET /identity/v3/users/618682c43f9c4ab29adf65cff57f6aa3 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:36.331092 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb3688-2f43-4f76-90f3-11b18df222d0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.331378 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb3688-2f43-4f76-90f3-11b18df222d0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.331656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb3688-2f43-4f76-90f3-11b18df222d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.349374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-13fb3688-2f43-4f76-90f3-11b18df222d0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:36.350640 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-13fb3688-2f43-4f76-90f3-11b18df222d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:36.392036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f1d9b-97b7-4595-a9e8-eadb87e3b93f None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:36.440706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 109/222] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:36.441744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb3688-2f43-4f76-90f3-11b18df222d0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:36.445630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1e1cf-a8eb-4e9e-887a-a34b07849458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:36.446644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1e1cf-a8eb-4e9e-887a-a34b07849458 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.447153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1e1cf-a8eb-4e9e-887a-a34b07849458 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.447621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1e1cf-a8eb-4e9e-887a-a34b07849458 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.448904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1e1cf-a8eb-4e9e-887a-a34b07849458 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:36.449408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1e1cf-a8eb-4e9e-887a-a34b07849458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:36.451519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1e1cf-a8eb-4e9e-887a-a34b07849458 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:36.461614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 114/223] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:36.467916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1e1cf-a8eb-4e9e-887a-a34b07849458 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:36.569447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:36.583234 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:36.595484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb3688-2f43-4f76-90f3-11b18df222d0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:36.595768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1e1cf-a8eb-4e9e-887a-a34b07849458 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:36.640898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de6e9e-e9b4-4270-b83b-ab9b5a20a0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:36.641563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de6e9e-e9b4-4270-b83b-ab9b5a20a0a3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.641847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de6e9e-e9b4-4270-b83b-ab9b5a20a0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.642171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de6e9e-e9b4-4270-b83b-ab9b5a20a0a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.644425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015713c-37da-465e-be00-702bfd4bcd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:36.645303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015713c-37da-465e-be00-702bfd4bcd7e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.645754 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015713c-37da-465e-be00-702bfd4bcd7e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.646047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015713c-37da-465e-be00-702bfd4bcd7e None admin] PATH_INFO: `/v3/users/e34570f1efd949dbb5edbf6f90a51dae` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.648120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015713c-37da-465e-be00-702bfd4bcd7e None admin] RBAC: Authorizing `identity:get_user(user_id=e34570f1efd949dbb5edbf6f90a51dae)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:36.649620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015713c-37da-465e-be00-702bfd4bcd7e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:36.651619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 115/224] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:36 2022] GET /identity/v3/users/e34570f1efd949dbb5edbf6f90a51dae => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:36.652553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015713c-37da-465e-be00-702bfd4bcd7e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:36.717351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de6e9e-e9b4-4270-b83b-ab9b5a20a0a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:36.718812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de6e9e-e9b4-4270-b83b-ab9b5a20a0a3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:36.762965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015713c-37da-465e-be00-702bfd4bcd7e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:36.798689 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0a55d-946f-419c-a5bf-2240735ced24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:36.799852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0a55d-946f-419c-a5bf-2240735ced24 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.799982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:36.800471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0a55d-946f-419c-a5bf-2240735ced24 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.800935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0a55d-946f-419c-a5bf-2240735ced24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.812164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0a55d-946f-419c-a5bf-2240735ced24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-37e0a55d-946f-419c-a5bf-2240735ced24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:36.820745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:36.821814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:36.822818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:36.824304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 116/225] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:36:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:36.825202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0a55d-946f-419c-a5bf-2240735ced24 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:36.828315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 110/226] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:36.830568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de6e9e-e9b4-4270-b83b-ab9b5a20a0a3 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:36.853410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:36.936417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0a55d-946f-419c-a5bf-2240735ced24 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:36.936787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de6e9e-e9b4-4270-b83b-ab9b5a20a0a3 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:36.972892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c46b7-09b9-462d-929c-5a0ed22e51b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:36.973553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c46b7-09b9-462d-929c-5a0ed22e51b3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.973835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c46b7-09b9-462d-929c-5a0ed22e51b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.974112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c46b7-09b9-462d-929c-5a0ed22e51b3 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.974575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2864e71-9cef-4248-a6b0-6495f692c8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:36.974941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c46b7-09b9-462d-929c-5a0ed22e51b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:36.975253 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c46b7-09b9-462d-929c-5a0ed22e51b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:36.976079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2864e71-9cef-4248-a6b0-6495f692c8d5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:36.976607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2864e71-9cef-4248-a6b0-6495f692c8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:36.976709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c46b7-09b9-462d-929c-5a0ed22e51b3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:36.977176 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2864e71-9cef-4248-a6b0-6495f692c8d5 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:36.978748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2864e71-9cef-4248-a6b0-6495f692c8d5 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:36.980451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2864e71-9cef-4248-a6b0-6495f692c8d5 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:36.982804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 117/227] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:36 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:36.983853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2864e71-9cef-4248-a6b0-6495f692c8d5 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:36.986113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 111/228] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:36:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:36.990858 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c46b7-09b9-462d-929c-5a0ed22e51b3 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:37.038239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:37.079302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:37.092156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2864e71-9cef-4248-a6b0-6495f692c8d5 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:37.105910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c46b7-09b9-462d-929c-5a0ed22e51b3 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:37.127766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fba3d-7340-411e-b679-ef0bccb7cec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:37.128683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fba3d-7340-411e-b679-ef0bccb7cec2 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:37.128971 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fba3d-7340-411e-b679-ef0bccb7cec2 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:37.129252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fba3d-7340-411e-b679-ef0bccb7cec2 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:37.130691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fba3d-7340-411e-b679-ef0bccb7cec2 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:37.131888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fba3d-7340-411e-b679-ef0bccb7cec2 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:37.133627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 118/229] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:36 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:37.134641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fba3d-7340-411e-b679-ef0bccb7cec2 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:37.146873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f66a2-5452-462e-81ba-1fbc18b1cda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:37.148125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f66a2-5452-462e-81ba-1fbc18b1cda9 None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:37.148746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f66a2-5452-462e-81ba-1fbc18b1cda9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:37.149350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f66a2-5452-462e-81ba-1fbc18b1cda9 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf/users/e34570f1efd949dbb5edbf6f90a51dae/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:37.151736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f66a2-5452-462e-81ba-1fbc18b1cda9 None admin] RBAC: Authorizing `identity:create_grant(project_id=682d8892f2d3403f862be249dc3a8acf, user_id=e34570f1efd949dbb5edbf6f90a51dae, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:37.153538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f66a2-5452-462e-81ba-1fbc18b1cda9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:37.220400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 112/230] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:36:36 2022] PUT /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf/users/e34570f1efd949dbb5edbf6f90a51dae/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:36:37.221709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f66a2-5452-462e-81ba-1fbc18b1cda9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:37.269622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fba3d-7340-411e-b679-ef0bccb7cec2 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:37.337729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db4b93-9d99-4cc0-bd38-57584472bd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:37.338646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db4b93-9d99-4cc0-bd38-57584472bd38 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:37.338933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db4b93-9d99-4cc0-bd38-57584472bd38 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:37.339210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db4b93-9d99-4cc0-bd38-57584472bd38 None admin] PATH_INFO: `/v3/users/e34570f1efd949dbb5edbf6f90a51dae` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:37.342224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db4b93-9d99-4cc0-bd38-57584472bd38 None admin] RBAC: Authorizing `identity:get_user(user_id=e34570f1efd949dbb5edbf6f90a51dae)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:37.343714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db4b93-9d99-4cc0-bd38-57584472bd38 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:37.345830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 119/231] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:37 2022] GET /identity/v3/users/e34570f1efd949dbb5edbf6f90a51dae => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:37.351339 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db4b93-9d99-4cc0-bd38-57584472bd38 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:37.391199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:37.464100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:37.466848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f66a2-5452-462e-81ba-1fbc18b1cda9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:37.487396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db4b93-9d99-4cc0-bd38-57584472bd38 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:37.508625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742746f-f7bb-4a30-8975-ca1841b7335c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:37.509566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742746f-f7bb-4a30-8975-ca1841b7335c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:37.509856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742746f-f7bb-4a30-8975-ca1841b7335c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:37.510139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742746f-f7bb-4a30-8975-ca1841b7335c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:37.510987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742746f-f7bb-4a30-8975-ca1841b7335c None admin] RBAC: Adding query filter params (role.id=177939861d6e4e82b48401be184d66b4, scope.project.id=682d8892f2d3403f862be249dc3a8acf, user.id=618682c43f9c4ab29adf65cff57f6aa3) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:37.511305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742746f-f7bb-4a30-8975-ca1841b7335c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:37.513129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742746f-f7bb-4a30-8975-ca1841b7335c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:37.519953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 113/232] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:37 2022] GET /identity/v3/role_assignments?user.id=618682c43f9c4ab29adf65cff57f6aa3&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=177939861d6e4e82b48401be184d66b4 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:37.528827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c208e-7f49-4741-9928-88bba95345f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:37.530083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c208e-7f49-4741-9928-88bba95345f6 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:37.530555 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c208e-7f49-4741-9928-88bba95345f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:37.531037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c208e-7f49-4741-9928-88bba95345f6 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:37.533546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c208e-7f49-4741-9928-88bba95345f6 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:37.535258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c208e-7f49-4741-9928-88bba95345f6 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:37.537711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 120/233] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:37 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:37.543053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742746f-f7bb-4a30-8975-ca1841b7335c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:37.653950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742746f-f7bb-4a30-8975-ca1841b7335c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:37.691374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f7382-1d23-442e-a6d6-11a7a9b58ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:37.692672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f7382-1d23-442e-a6d6-11a7a9b58ba1 None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:37.693153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f7382-1d23-442e-a6d6-11a7a9b58ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:37.693603 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f7382-1d23-442e-a6d6-11a7a9b58ba1 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf/users/e34570f1efd949dbb5edbf6f90a51dae/roles/f7e0a506c7894b20b2b65fedfd2244e5` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:37.696123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f7382-1d23-442e-a6d6-11a7a9b58ba1 None admin] RBAC: Authorizing `identity:create_grant(project_id=682d8892f2d3403f862be249dc3a8acf, user_id=e34570f1efd949dbb5edbf6f90a51dae, role_id=f7e0a506c7894b20b2b65fedfd2244e5)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:37.697896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f7382-1d23-442e-a6d6-11a7a9b58ba1 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:37.814493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 114/234] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:36:37 2022] PUT /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf/users/e34570f1efd949dbb5edbf6f90a51dae/roles/f7e0a506c7894b20b2b65fedfd2244e5 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:36:38.550755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b576f-6a43-4037-88f9-6daeb77a92e9 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:38.551654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b576f-6a43-4037-88f9-6daeb77a92e9 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:38.552112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b576f-6a43-4037-88f9-6daeb77a92e9 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:38.553939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 121/235] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:38.559495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c321276-af61-4ed8-9d26-82d59afde648 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:38.559974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c321276-af61-4ed8-9d26-82d59afde648 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:38.560511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c321276-af61-4ed8-9d26-82d59afde648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:38.584219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2c321276-af61-4ed8-9d26-82d59afde648 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:38.585905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2c321276-af61-4ed8-9d26-82d59afde648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:38.610873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:38.686801 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 115/236] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:38.725867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b576f-6a43-4037-88f9-6daeb77a92e9 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:38.859928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:38.870204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b576f-6a43-4037-88f9-6daeb77a92e9 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:38.891238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1914b-8a25-42a9-ba49-724d3ff9eb42 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:38.891534 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1914b-8a25-42a9-ba49-724d3ff9eb42 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:38.891816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1914b-8a25-42a9-ba49-724d3ff9eb42 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:38.893428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 116/237] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:38.899480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7964f-94e1-4511-b7a3-7437a39f20be None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:38.899765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7964f-94e1-4511-b7a3-7437a39f20be None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:38.900045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7964f-94e1-4511-b7a3-7437a39f20be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:38.927907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf951f-e919-4742-a833-17bad0577891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:38.929107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf951f-e919-4742-a833-17bad0577891 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:38.929613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf951f-e919-4742-a833-17bad0577891 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:38.930083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf951f-e919-4742-a833-17bad0577891 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:38.940041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-5bb7964f-94e1-4511-b7a3-7437a39f20be None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:38.942036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5bb7964f-94e1-4511-b7a3-7437a39f20be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:38.951630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf951f-e919-4742-a833-17bad0577891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-43cf951f-e919-4742-a833-17bad0577891 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:38.964293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:38.965192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:38.965997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:38.976837 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 122/238] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:38.981631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf951f-e919-4742-a833-17bad0577891 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:39.100587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf951f-e919-4742-a833-17bad0577891 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:39.117393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 117/239] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:39.150672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa831373-9df5-4fa0-a028-eb1493b04d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:39.151601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa831373-9df5-4fa0-a028-eb1493b04d81 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:39.152051 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa831373-9df5-4fa0-a028-eb1493b04d81 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:39.159669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa831373-9df5-4fa0-a028-eb1493b04d81 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:39.159669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa831373-9df5-4fa0-a028-eb1493b04d81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:39.159669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa831373-9df5-4fa0-a028-eb1493b04d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:39.159669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa831373-9df5-4fa0-a028-eb1493b04d81 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:39.168145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 123/240] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:39.173409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7964f-94e1-4511-b7a3-7437a39f20be None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:39.178220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa831373-9df5-4fa0-a028-eb1493b04d81 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:39.202986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:39.296461 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa831373-9df5-4fa0-a028-eb1493b04d81 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:39.299984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:39.316217 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7964f-94e1-4511-b7a3-7437a39f20be None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:39.341832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf645c4b-5a15-4494-a0ee-512699edc791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:39.342646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf645c4b-5a15-4494-a0ee-512699edc791 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:39.342932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf645c4b-5a15-4494-a0ee-512699edc791 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:39.343210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf645c4b-5a15-4494-a0ee-512699edc791 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:39.349268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf645c4b-5a15-4494-a0ee-512699edc791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:39.350679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115050c-1604-436b-b3c9-ff208bf5fc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:39.350802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf645c4b-5a15-4494-a0ee-512699edc791 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:39.351580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115050c-1604-436b-b3c9-ff208bf5fc94 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:39.351866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115050c-1604-436b-b3c9-ff208bf5fc94 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:39.352144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115050c-1604-436b-b3c9-ff208bf5fc94 None admin] PATH_INFO: `/v3/users/156cca94b61c4cf18b45104afc32a9df` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:39.352521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 124/241] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:39.358874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf645c4b-5a15-4494-a0ee-512699edc791 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:39.369641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115050c-1604-436b-b3c9-ff208bf5fc94 None admin] RBAC: Authorizing `identity:get_user(user_id=156cca94b61c4cf18b45104afc32a9df)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:39.371109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115050c-1604-436b-b3c9-ff208bf5fc94 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:39.373725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 118/242] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:39 2022] GET /identity/v3/users/156cca94b61c4cf18b45104afc32a9df => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:39.378984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115050c-1604-436b-b3c9-ff208bf5fc94 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:39.478925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115050c-1604-436b-b3c9-ff208bf5fc94 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:39.514193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde677e-13e0-4716-920b-2dddf19cac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:39.514394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:39.514856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde677e-13e0-4716-920b-2dddf19cac19 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:39.515144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde677e-13e0-4716-920b-2dddf19cac19 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:39.515429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde677e-13e0-4716-920b-2dddf19cac19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:39.531898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:39.581394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf645c4b-5a15-4494-a0ee-512699edc791 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:39.583931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde677e-13e0-4716-920b-2dddf19cac19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:39.585410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde677e-13e0-4716-920b-2dddf19cac19 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:39.705253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a871e8e-b2f7-4b62-bf59-16de62d35e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:39.705870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a871e8e-b2f7-4b62-bf59-16de62d35e6f None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:39.706153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a871e8e-b2f7-4b62-bf59-16de62d35e6f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:39.709457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a871e8e-b2f7-4b62-bf59-16de62d35e6f None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:39.709457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a871e8e-b2f7-4b62-bf59-16de62d35e6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:39.709789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a871e8e-b2f7-4b62-bf59-16de62d35e6f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:39.765775 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 119/243] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:39.772144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde677e-13e0-4716-920b-2dddf19cac19 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:39.822627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 125/244] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:36:39 2022] POST /identity/v3/users => generated 293 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:39.822905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:39.945919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde677e-13e0-4716-920b-2dddf19cac19 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:40.035153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3bf78-0a1b-4773-85e4-a433bf5995d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:40.036035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3bf78-0a1b-4773-85e4-a433bf5995d3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.036312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3bf78-0a1b-4773-85e4-a433bf5995d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.036606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3bf78-0a1b-4773-85e4-a433bf5995d3 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.037807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3bf78-0a1b-4773-85e4-a433bf5995d3 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:40.039232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3bf78-0a1b-4773-85e4-a433bf5995d3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:40.049530 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 120/245] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:39 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:40.057610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a871e8e-b2f7-4b62-bf59-16de62d35e6f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:40.105476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f1a22-ba1f-40a5-accd-bb5628da1c43 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.105757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f1a22-ba1f-40a5-accd-bb5628da1c43 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.106030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f1a22-ba1f-40a5-accd-bb5628da1c43 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.107510 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 121/246] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:40.113728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c4979-62c7-4c89-8160-96e90827a25f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.114015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c4979-62c7-4c89-8160-96e90827a25f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.114287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c4979-62c7-4c89-8160-96e90827a25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.147884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-cd2c4979-62c7-4c89-8160-96e90827a25f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:40.149128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-cd2c4979-62c7-4c89-8160-96e90827a25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:40.249959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 122/247] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:40.251419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f484b6-7479-4374-a119-ce174c9f37e8 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.251712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f484b6-7479-4374-a119-ce174c9f37e8 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.251989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f484b6-7479-4374-a119-ce174c9f37e8 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.253079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a871e8e-b2f7-4b62-bf59-16de62d35e6f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:40.253282 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 123/248] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:40.260702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa4939-7a6f-4489-ace7-7962a10d9b0b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.260702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa4939-7a6f-4489-ace7-7962a10d9b0b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.260702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa4939-7a6f-4489-ace7-7962a10d9b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.281949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c6fa4939-7a6f-4489-ace7-7962a10d9b0b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:40.283762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c6fa4939-7a6f-4489-ace7-7962a10d9b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:40.298322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33389d-e0c3-4945-a975-e60678f39a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:40.298997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33389d-e0c3-4945-a975-e60678f39a24 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.299282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33389d-e0c3-4945-a975-e60678f39a24 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.299559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33389d-e0c3-4945-a975-e60678f39a24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.300533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33389d-e0c3-4945-a975-e60678f39a24 None admin] RBAC: Adding query filter params (role.id=2a5bf966af5a486fa21c10760ebfbcad, scope.project.id=682d8892f2d3403f862be249dc3a8acf, user.id=156cca94b61c4cf18b45104afc32a9df) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:40.300851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33389d-e0c3-4945-a975-e60678f39a24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:40.302302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33389d-e0c3-4945-a975-e60678f39a24 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:40.308582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 126/249] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:40 2022] GET /identity/v3/role_assignments?user.id=156cca94b61c4cf18b45104afc32a9df&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=2a5bf966af5a486fa21c10760ebfbcad => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:40.312712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33389d-e0c3-4945-a975-e60678f39a24 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:40.327437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:40.329637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:40.366806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 124/250] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:40.390306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa4939-7a6f-4489-ace7-7962a10d9b0b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:40.437937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33389d-e0c3-4945-a975-e60678f39a24 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:40.476816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b2bf-cca4-4479-9935-2047694605cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:40.477736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b2bf-cca4-4479-9935-2047694605cd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.478085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b2bf-cca4-4479-9935-2047694605cd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.478529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b2bf-cca4-4479-9935-2047694605cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.491528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517b2bf-cca4-4479-9935-2047694605cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-9517b2bf-cca4-4479-9935-2047694605cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:40.500831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:40.501811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:40.502594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 127/251] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:36:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:40.512932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b2bf-cca4-4479-9935-2047694605cd None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:40.530783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa4939-7a6f-4489-ace7-7962a10d9b0b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:40.573959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e74b07-6b38-4896-a077-e34ce4acfe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:40.575105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e74b07-6b38-4896-a077-e34ce4acfe59 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.575590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e74b07-6b38-4896-a077-e34ce4acfe59 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.576647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e74b07-6b38-4896-a077-e34ce4acfe59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.586544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:40.590765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e74b07-6b38-4896-a077-e34ce4acfe59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-d1e74b07-6b38-4896-a077-e34ce4acfe59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:40.600155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:40.601100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:40.601888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:40.608711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 125/252] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:40.610964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e74b07-6b38-4896-a077-e34ce4acfe59 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:40.630099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b2bf-cca4-4479-9935-2047694605cd None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:40.649345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:40.668442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fc69d-775c-4d2c-95b6-e54669f31d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:40.669201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fc69d-775c-4d2c-95b6-e54669f31d51 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.669490 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fc69d-775c-4d2c-95b6-e54669f31d51 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.669799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fc69d-775c-4d2c-95b6-e54669f31d51 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.670558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750fc69d-775c-4d2c-95b6-e54669f31d51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:40.670869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750fc69d-775c-4d2c-95b6-e54669f31d51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:40.672258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750fc69d-775c-4d2c-95b6-e54669f31d51 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:40.682184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 128/253] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:40.688570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fc69d-775c-4d2c-95b6-e54669f31d51 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:40.727871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e74b07-6b38-4896-a077-e34ce4acfe59 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:40.767621 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbedf9-47f4-40f3-97cd-5b7a2c4965e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:40.769077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbedf9-47f4-40f3-97cd-5b7a2c4965e8 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.769541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbedf9-47f4-40f3-97cd-5b7a2c4965e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.770086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbedf9-47f4-40f3-97cd-5b7a2c4965e8 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.771047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbedf9-47f4-40f3-97cd-5b7a2c4965e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:40.771550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbedf9-47f4-40f3-97cd-5b7a2c4965e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:40.773206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbedf9-47f4-40f3-97cd-5b7a2c4965e8 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:40.782874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 126/254] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:40.788690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6325cdd-cdaa-4c3c-b59f-7bc57ea58626 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.789338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6325cdd-cdaa-4c3c-b59f-7bc57ea58626 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.789827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6325cdd-cdaa-4c3c-b59f-7bc57ea58626 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.791473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 127/255] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:40.793075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6325cdd-cdaa-4c3c-b59f-7bc57ea58626 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:40.799408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fc69d-775c-4d2c-95b6-e54669f31d51 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:40.832882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:40.836049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e33d1-b936-4912-97d6-9bc5ace6fbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:40.837068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e33d1-b936-4912-97d6-9bc5ace6fbe9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.837354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e33d1-b936-4912-97d6-9bc5ace6fbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.837631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e33d1-b936-4912-97d6-9bc5ace6fbe9 None admin] PATH_INFO: `/v3/users/e34570f1efd949dbb5edbf6f90a51dae` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.838939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e33d1-b936-4912-97d6-9bc5ace6fbe9 None admin] RBAC: Authorizing `identity:get_user(user_id=e34570f1efd949dbb5edbf6f90a51dae)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:40.840496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e33d1-b936-4912-97d6-9bc5ace6fbe9 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:40.842510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 129/256] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:40 2022] GET /identity/v3/users/e34570f1efd949dbb5edbf6f90a51dae => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:40.843898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c661191-7144-44ad-a8e9-494e4fdff077 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:40.844205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c661191-7144-44ad-a8e9-494e4fdff077 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:40.849071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c661191-7144-44ad-a8e9-494e4fdff077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:40.867302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0c661191-7144-44ad-a8e9-494e4fdff077 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:40.868609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0c661191-7144-44ad-a8e9-494e4fdff077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:40.938769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 130/257] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:40.939563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c661191-7144-44ad-a8e9-494e4fdff077 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:40.977732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:41.017030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6325cdd-cdaa-4c3c-b59f-7bc57ea58626 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:41.043391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c661191-7144-44ad-a8e9-494e4fdff077 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:41.068876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:41.079597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd03a2-eac4-4cd9-9891-034f9bc27d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:41.080266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd03a2-eac4-4cd9-9891-034f9bc27d21 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:41.080574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd03a2-eac4-4cd9-9891-034f9bc27d21 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:41.080855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd03a2-eac4-4cd9-9891-034f9bc27d21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:41.130984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7905d1-1933-45d2-b503-a8e233da54be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:41.131787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7905d1-1933-45d2-b503-a8e233da54be None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:41.132073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7905d1-1933-45d2-b503-a8e233da54be None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:41.132343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7905d1-1933-45d2-b503-a8e233da54be None admin] PATH_INFO: `/v3/users/618682c43f9c4ab29adf65cff57f6aa3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:41.133598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7905d1-1933-45d2-b503-a8e233da54be None admin] RBAC: Authorizing `identity:get_user(user_id=618682c43f9c4ab29adf65cff57f6aa3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:41.135441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7905d1-1933-45d2-b503-a8e233da54be None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:41.137105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 128/258] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:40 2022] GET /identity/v3/users/618682c43f9c4ab29adf65cff57f6aa3 => generated 297 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:41.137876 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7905d1-1933-45d2-b503-a8e233da54be None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:41.156486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd03a2-eac4-4cd9-9891-034f9bc27d21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:41.157966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd03a2-eac4-4cd9-9891-034f9bc27d21 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:41.261495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 131/259] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:41.262359 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd03a2-eac4-4cd9-9891-034f9bc27d21 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:41.304751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:41.376779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd03a2-eac4-4cd9-9891-034f9bc27d21 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:41.380845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7905d1-1933-45d2-b503-a8e233da54be None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:41.413045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38038e-8d12-456f-b47c-52e3218bc8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:41.413960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38038e-8d12-456f-b47c-52e3218bc8ed None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:41.414238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38038e-8d12-456f-b47c-52e3218bc8ed None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:41.414509 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38038e-8d12-456f-b47c-52e3218bc8ed None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:41.415774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38038e-8d12-456f-b47c-52e3218bc8ed None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:41.417250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38038e-8d12-456f-b47c-52e3218bc8ed None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:41.419132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 132/260] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:41 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:41.420516 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38038e-8d12-456f-b47c-52e3218bc8ed None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:41.447898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b1430-e1ab-411e-81bc-96d00129f76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:41.449190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b1430-e1ab-411e-81bc-96d00129f76a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:41.449724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b1430-e1ab-411e-81bc-96d00129f76a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:41.450300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b1430-e1ab-411e-81bc-96d00129f76a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:41.477362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b1430-e1ab-411e-81bc-96d00129f76a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-e77b1430-e1ab-411e-81bc-96d00129f76a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:41.493265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:41.494165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:41.494946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:41.497706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 129/261] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:36:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:41.499347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b1430-e1ab-411e-81bc-96d00129f76a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:41.520749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:41.526404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38038e-8d12-456f-b47c-52e3218bc8ed None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:41.544384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:41.571530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3977f-b752-4625-91c8-fea16586a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:41.576560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3977f-b752-4625-91c8-fea16586a35d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:41.576867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3977f-b752-4625-91c8-fea16586a35d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:41.577155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3977f-b752-4625-91c8-fea16586a35d None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:41.578362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3977f-b752-4625-91c8-fea16586a35d None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:41.579805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3977f-b752-4625-91c8-fea16586a35d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:41.581511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 133/262] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:41 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:41.582262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3977f-b752-4625-91c8-fea16586a35d None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:41.632431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b1430-e1ab-411e-81bc-96d00129f76a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:41.687150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e2d57-320e-4436-8779-0227b4a8565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:41.689162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e2d57-320e-4436-8779-0227b4a8565a None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:41.689805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e2d57-320e-4436-8779-0227b4a8565a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:41.691181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e2d57-320e-4436-8779-0227b4a8565a None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf/users/618682c43f9c4ab29adf65cff57f6aa3/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:41.694241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e2d57-320e-4436-8779-0227b4a8565a None admin] RBAC: Authorizing `identity:create_grant(project_id=682d8892f2d3403f862be249dc3a8acf, user_id=618682c43f9c4ab29adf65cff57f6aa3, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:41.696508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e2d57-320e-4436-8779-0227b4a8565a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:41.739558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3977f-b752-4625-91c8-fea16586a35d None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:41.780208 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd3a72-0800-4171-80d1-4f9583bfa2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:41.780954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd3a72-0800-4171-80d1-4f9583bfa2ac None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:41.781245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd3a72-0800-4171-80d1-4f9583bfa2ac None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:41.781528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd3a72-0800-4171-80d1-4f9583bfa2ac None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:41.782331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd3a72-0800-4171-80d1-4f9583bfa2ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:41.782643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd3a72-0800-4171-80d1-4f9583bfa2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:41.784142 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd3a72-0800-4171-80d1-4f9583bfa2ac None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:41.792391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 134/263] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:36:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:41.793410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd3a72-0800-4171-80d1-4f9583bfa2ac None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:41.818488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:41.827548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 130/264] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:36:41 2022] PUT /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf/users/618682c43f9c4ab29adf65cff57f6aa3/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:41.829225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e2d57-320e-4436-8779-0227b4a8565a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:41.975343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd3a72-0800-4171-80d1-4f9583bfa2ac None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:41.976780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:41.997237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e2d57-320e-4436-8779-0227b4a8565a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:42.014924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c64d7-6b5e-4e23-a1cc-f47eb903ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:42.015902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c64d7-6b5e-4e23-a1cc-f47eb903ff1e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:42.016325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c64d7-6b5e-4e23-a1cc-f47eb903ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:42.016902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c64d7-6b5e-4e23-a1cc-f47eb903ff1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:42.017980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c64d7-6b5e-4e23-a1cc-f47eb903ff1e None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=682d8892f2d3403f862be249dc3a8acf, user.id=e34570f1efd949dbb5edbf6f90a51dae) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:42.018455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c64d7-6b5e-4e23-a1cc-f47eb903ff1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:42.020091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c64d7-6b5e-4e23-a1cc-f47eb903ff1e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:42.026973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 135/265] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:41 2022] GET /identity/v3/role_assignments?user.id=e34570f1efd949dbb5edbf6f90a51dae&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=04c11dea81274a69974ad5c639f2c60b => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:42.039508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427c09d-1dd8-464e-a4bb-5512e1f64945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:42.044218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427c09d-1dd8-464e-a4bb-5512e1f64945 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:42.044218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427c09d-1dd8-464e-a4bb-5512e1f64945 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:42.044218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427c09d-1dd8-464e-a4bb-5512e1f64945 None admin] PATH_INFO: `/v3/users/e34570f1efd949dbb5edbf6f90a51dae` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:42.044218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427c09d-1dd8-464e-a4bb-5512e1f64945 None admin] RBAC: Authorizing `identity:get_user(user_id=e34570f1efd949dbb5edbf6f90a51dae)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:42.044218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427c09d-1dd8-464e-a4bb-5512e1f64945 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:42.047178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 131/266] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:41 2022] GET /identity/v3/users/e34570f1efd949dbb5edbf6f90a51dae => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:42.053164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c64d7-6b5e-4e23-a1cc-f47eb903ff1e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:42.182286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:42.198700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c64d7-6b5e-4e23-a1cc-f47eb903ff1e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:42.248964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e7333-bff8-4cae-8b77-2c25ecb78a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:42.249991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e7333-bff8-4cae-8b77-2c25ecb78a41 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:42.250469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e7333-bff8-4cae-8b77-2c25ecb78a41 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:42.250932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e7333-bff8-4cae-8b77-2c25ecb78a41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:42.333620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e7333-bff8-4cae-8b77-2c25ecb78a41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:42.335124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e7333-bff8-4cae-8b77-2c25ecb78a41 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:42.425286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:42.442282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 136/267] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:42.447658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427c09d-1dd8-464e-a4bb-5512e1f64945 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:42.606574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427c09d-1dd8-464e-a4bb-5512e1f64945 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:42.648610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de8602-8229-4005-946d-cc8aff281152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:42.649481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de8602-8229-4005-946d-cc8aff281152 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:42.649761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de8602-8229-4005-946d-cc8aff281152 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:42.650037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de8602-8229-4005-946d-cc8aff281152 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:42.651228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de8602-8229-4005-946d-cc8aff281152 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:42.652692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de8602-8229-4005-946d-cc8aff281152 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:42.655259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 132/268] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:42 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:42.660892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e7333-bff8-4cae-8b77-2c25ecb78a41 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:42.809033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e7333-bff8-4cae-8b77-2c25ecb78a41 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:42.853854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d819a-efe1-434b-9871-6141d26379dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:42.854524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d819a-efe1-434b-9871-6141d26379dd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:42.854810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d819a-efe1-434b-9871-6141d26379dd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:42.855094 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d819a-efe1-434b-9871-6141d26379dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:42.855924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d819a-efe1-434b-9871-6141d26379dd None admin] RBAC: Adding query filter params (role.id=f7e0a506c7894b20b2b65fedfd2244e5, scope.project.id=682d8892f2d3403f862be249dc3a8acf, user.id=e34570f1efd949dbb5edbf6f90a51dae) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:42.856241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d819a-efe1-434b-9871-6141d26379dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:42.857806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d819a-efe1-434b-9871-6141d26379dd None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:42.869196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 137/269] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:42 2022] GET /identity/v3/role_assignments?user.id=e34570f1efd949dbb5edbf6f90a51dae&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=f7e0a506c7894b20b2b65fedfd2244e5 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:43.270278 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff18a-3742-4cce-9fb3-40f854a25dc5 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.271309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff18a-3742-4cce-9fb3-40f854a25dc5 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.271799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff18a-3742-4cce-9fb3-40f854a25dc5 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.273429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 133/270] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:43.289760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e03b4-97e1-4316-85b6-21e8c0bce5ae None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.290267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e03b4-97e1-4316-85b6-21e8c0bce5ae None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.290742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e03b4-97e1-4316-85b6-21e8c0bce5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.313303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-bf8e03b4-97e1-4316-85b6-21e8c0bce5ae None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:43.314496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bf8e03b4-97e1-4316-85b6-21e8c0bce5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:43.328028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a02e2-1a22-4219-bb21-6b52b64b45d9 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.328635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a02e2-1a22-4219-bb21-6b52b64b45d9 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.329120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a02e2-1a22-4219-bb21-6b52b64b45d9 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.330688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 134/271] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:43.335954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd729d53-bb69-42ec-8e5d-f9c3f9afe0a1 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.336503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd729d53-bb69-42ec-8e5d-f9c3f9afe0a1 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.336976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd729d53-bb69-42ec-8e5d-f9c3f9afe0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.357018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-dd729d53-bb69-42ec-8e5d-f9c3f9afe0a1 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:43.358900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dd729d53-bb69-42ec-8e5d-f9c3f9afe0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:43.367367 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:43.370546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:43.393157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 138/272] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:43.394861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843edd64-138f-4928-81a8-089f0eae592e None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.395123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843edd64-138f-4928-81a8-089f0eae592e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.395405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843edd64-138f-4928-81a8-089f0eae592e None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.396754 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 139/273] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:43.402116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caace6c9-11e9-4ed3-b89b-5cbac08db911 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.402412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caace6c9-11e9-4ed3-b89b-5cbac08db911 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.402700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caace6c9-11e9-4ed3-b89b-5cbac08db911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.426279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-caace6c9-11e9-4ed3-b89b-5cbac08db911 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:43.427888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-caace6c9-11e9-4ed3-b89b-5cbac08db911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:43.455163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 135/274] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:43.455163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd729d53-bb69-42ec-8e5d-f9c3f9afe0a1 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:43.524451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 140/275] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:43.524451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caace6c9-11e9-4ed3-b89b-5cbac08db911 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:43.578126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd729d53-bb69-42ec-8e5d-f9c3f9afe0a1 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:43.618903 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e4073-6b11-4346-a33f-1e9f0314dab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:43.619702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e4073-6b11-4346-a33f-1e9f0314dab5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.619986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e4073-6b11-4346-a33f-1e9f0314dab5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.620259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e4073-6b11-4346-a33f-1e9f0314dab5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.621455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e4073-6b11-4346-a33f-1e9f0314dab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:43.622931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e4073-6b11-4346-a33f-1e9f0314dab5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:43.624739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 136/276] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:43.625338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e4073-6b11-4346-a33f-1e9f0314dab5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:43.633290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caace6c9-11e9-4ed3-b89b-5cbac08db911 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:43.641427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:43.644217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:43.671250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd90b42-0d80-43d5-95b9-1d6bf352eab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:43.672376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd90b42-0d80-43d5-95b9-1d6bf352eab3 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.672748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd90b42-0d80-43d5-95b9-1d6bf352eab3 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.673186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd90b42-0d80-43d5-95b9-1d6bf352eab3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.683273 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd90b42-0d80-43d5-95b9-1d6bf352eab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-6cd90b42-0d80-43d5-95b9-1d6bf352eab3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:43.690936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:43.691881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:43.692709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 141/277] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:43.693730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd90b42-0d80-43d5-95b9-1d6bf352eab3 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:43.743063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e4073-6b11-4346-a33f-1e9f0314dab5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:43.777547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a366e0d-e2ce-401b-aa60-e7fc075ca237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:43.778273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366e0d-e2ce-401b-aa60-e7fc075ca237 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.778558 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366e0d-e2ce-401b-aa60-e7fc075ca237 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.778837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366e0d-e2ce-401b-aa60-e7fc075ca237 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.790520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a366e0d-e2ce-401b-aa60-e7fc075ca237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-5a366e0d-e2ce-401b-aa60-e7fc075ca237 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:43.798270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:43.799371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:43.800195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 137/278] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:43.800864 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a366e0d-e2ce-401b-aa60-e7fc075ca237 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:43.807950 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd90b42-0d80-43d5-95b9-1d6bf352eab3 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:43.842314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f4cf4-f808-4b1d-83f0-023e7a808ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:43.843162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f4cf4-f808-4b1d-83f0-023e7a808ab3 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.843275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f4cf4-f808-4b1d-83f0-023e7a808ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.843859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f4cf4-f808-4b1d-83f0-023e7a808ab3 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.844225 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f4cf4-f808-4b1d-83f0-023e7a808ab3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:43.845752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f4cf4-f808-4b1d-83f0-023e7a808ab3 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:43.879684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:43.905258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a366e0d-e2ce-401b-aa60-e7fc075ca237 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:43.937454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05ea9b-3ab6-4a8a-8e3d-6e23cf6df848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:43.938189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05ea9b-3ab6-4a8a-8e3d-6e23cf6df848 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:43.938473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05ea9b-3ab6-4a8a-8e3d-6e23cf6df848 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:43.938755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05ea9b-3ab6-4a8a-8e3d-6e23cf6df848 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:43.939513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05ea9b-3ab6-4a8a-8e3d-6e23cf6df848 None admin] RBAC: Adding query filter params (name=member) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:43.944675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 142/279] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:36:43 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:43.944841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05ea9b-3ab6-4a8a-8e3d-6e23cf6df848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:43.944940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f4cf4-f808-4b1d-83f0-023e7a808ab3 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:43.945014 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05ea9b-3ab6-4a8a-8e3d-6e23cf6df848 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:43.950920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 138/280] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:43.957300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05ea9b-3ab6-4a8a-8e3d-6e23cf6df848 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:43.990385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:44.077023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05ea9b-3ab6-4a8a-8e3d-6e23cf6df848 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:44.088675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f4cf4-f808-4b1d-83f0-023e7a808ab3 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:44.115244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa6152-9aa0-432e-84e2-67a64dc45224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:44.116137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa6152-9aa0-432e-84e2-67a64dc45224 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.116469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa6152-9aa0-432e-84e2-67a64dc45224 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.116747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa6152-9aa0-432e-84e2-67a64dc45224 None admin] PATH_INFO: `/v3/users/50f37600e12e469aa43fdb6e3ac483b3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.119955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:44.128172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd2af8-6128-4584-9a0e-ef928d2c2ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:44.128895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd2af8-6128-4584-9a0e-ef928d2c2ff6 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.129184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd2af8-6128-4584-9a0e-ef928d2c2ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.129462 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd2af8-6128-4584-9a0e-ef928d2c2ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.130228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd2af8-6128-4584-9a0e-ef928d2c2ff6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:44.130540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd2af8-6128-4584-9a0e-ef928d2c2ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:44.131972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd2af8-6128-4584-9a0e-ef928d2c2ff6 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:44.132613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa6152-9aa0-432e-84e2-67a64dc45224 None admin] RBAC: Authorizing `identity:get_user(user_id=50f37600e12e469aa43fdb6e3ac483b3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:44.134571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa6152-9aa0-432e-84e2-67a64dc45224 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:44.141882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 139/281] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:43 2022] GET /identity/v3/users/50f37600e12e469aa43fdb6e3ac483b3 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:44.142812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa6152-9aa0-432e-84e2-67a64dc45224 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:44.143465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 143/282] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:44.148991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd2af8-6128-4584-9a0e-ef928d2c2ff6 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:44.260032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:44.269692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa6152-9aa0-432e-84e2-67a64dc45224 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:44.276235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd2af8-6128-4584-9a0e-ef928d2c2ff6 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:44.311790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec49d1-eb41-4340-8526-64fd943ea6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:44.313107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec49d1-eb41-4340-8526-64fd943ea6f9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.314497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec49d1-eb41-4340-8526-64fd943ea6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.314497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec49d1-eb41-4340-8526-64fd943ea6f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.314497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec49d1-eb41-4340-8526-64fd943ea6f9 None admin] RBAC: Adding query filter params (role.id=177939861d6e4e82b48401be184d66b4, user.id=50f37600e12e469aa43fdb6e3ac483b3, scope.system=all) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:44.314829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec49d1-eb41-4340-8526-64fd943ea6f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:44.316257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec49d1-eb41-4340-8526-64fd943ea6f9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:44.327419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 140/283] 10.222.0.23 () {56 vars in 1308 bytes} [Mon May 23 15:36:44 2022] GET /identity/v3/role_assignments?user.id=50f37600e12e469aa43fdb6e3ac483b3&scope.system=all&role.id=177939861d6e4e82b48401be184d66b4 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:44.331148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5110881-08ba-46d5-b30d-b094cbf5a2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:44.332158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5110881-08ba-46d5-b30d-b094cbf5a2dd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.332536 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5110881-08ba-46d5-b30d-b094cbf5a2dd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.332883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5110881-08ba-46d5-b30d-b094cbf5a2dd None admin] PATH_INFO: `/v3/users/156cca94b61c4cf18b45104afc32a9df` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.334324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5110881-08ba-46d5-b30d-b094cbf5a2dd None admin] RBAC: Authorizing `identity:get_user(user_id=156cca94b61c4cf18b45104afc32a9df)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:44.335904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5110881-08ba-46d5-b30d-b094cbf5a2dd None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:44.338124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 144/284] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:44 2022] GET /identity/v3/users/156cca94b61c4cf18b45104afc32a9df => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:44.344258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec49d1-eb41-4340-8526-64fd943ea6f9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:44.420177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:44.469518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec49d1-eb41-4340-8526-64fd943ea6f9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:44.515778 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5337aea-a701-4c97-ba5f-89912bb7102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:44.517087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5337aea-a701-4c97-ba5f-89912bb7102f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.517577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5337aea-a701-4c97-ba5f-89912bb7102f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.518089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5337aea-a701-4c97-ba5f-89912bb7102f None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.519689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5337aea-a701-4c97-ba5f-89912bb7102f None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:44.521430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5337aea-a701-4c97-ba5f-89912bb7102f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:44.523736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 141/285] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:44 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:44.529287 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5110881-08ba-46d5-b30d-b094cbf5a2dd None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:44.602336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369107d-f193-428e-ad2d-303b3c55d62f None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.602644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369107d-f193-428e-ad2d-303b3c55d62f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.602850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369107d-f193-428e-ad2d-303b3c55d62f None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.604218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 142/286] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:44.609815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fd2c7-35b0-47d1-961e-e833c6e213c6 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.610101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fd2c7-35b0-47d1-961e-e833c6e213c6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.610382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fd2c7-35b0-47d1-961e-e833c6e213c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.635122 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-d29fd2c7-35b0-47d1-961e-e833c6e213c6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:44.636310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d29fd2c7-35b0-47d1-961e-e833c6e213c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:44.676187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5110881-08ba-46d5-b30d-b094cbf5a2dd None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:44.726913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b16b0-9979-4cb4-aed5-31e699f81045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:44.727818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b16b0-9979-4cb4-aed5-31e699f81045 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.728099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b16b0-9979-4cb4-aed5-31e699f81045 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.728391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b16b0-9979-4cb4-aed5-31e699f81045 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf/users/156cca94b61c4cf18b45104afc32a9df/roles/2a5bf966af5a486fa21c10760ebfbcad` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.733289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b16b0-9979-4cb4-aed5-31e699f81045 None admin] RBAC: Authorizing `identity:create_grant(project_id=682d8892f2d3403f862be249dc3a8acf, user_id=156cca94b61c4cf18b45104afc32a9df, role_id=2a5bf966af5a486fa21c10760ebfbcad)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:44.737650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b16b0-9979-4cb4-aed5-31e699f81045 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:44.758100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 143/287] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:44.782547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29fd2c7-35b0-47d1-961e-e833c6e213c6 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:44.824534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 145/288] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:36:44 2022] PUT /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf/users/156cca94b61c4cf18b45104afc32a9df/roles/2a5bf966af5a486fa21c10760ebfbcad => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:44.859922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:44.919757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29fd2c7-35b0-47d1-961e-e833c6e213c6 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:44.958308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f39b6a-e132-439c-b8ba-c8e6844111f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:44.959121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f39b6a-e132-439c-b8ba-c8e6844111f9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:44.959401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f39b6a-e132-439c-b8ba-c8e6844111f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:44.959675 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f39b6a-e132-439c-b8ba-c8e6844111f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:44.971165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f39b6a-e132-439c-b8ba-c8e6844111f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-29f39b6a-e132-439c-b8ba-c8e6844111f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:44.979189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:44.980129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:44.981037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 144/289] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:44.987718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b16b0-9979-4cb4-aed5-31e699f81045 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.010168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:45.107788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b16b0-9979-4cb4-aed5-31e699f81045 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.150029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fad2e-e649-4c06-8f22-765fa275ec6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.150706 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fad2e-e649-4c06-8f22-765fa275ec6c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.150998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fad2e-e649-4c06-8f22-765fa275ec6c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.151278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fad2e-e649-4c06-8f22-765fa275ec6c None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.152044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fad2e-e649-4c06-8f22-765fa275ec6c None admin] RBAC: Adding query filter params (name=member) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:45.152377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fad2e-e649-4c06-8f22-765fa275ec6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.153767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fad2e-e649-4c06-8f22-765fa275ec6c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.163893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 146/290] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:45.169432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f39b6a-e132-439c-b8ba-c8e6844111f9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.298271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f39b6a-e132-439c-b8ba-c8e6844111f9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.315491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:45.333159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70244f24-0a45-495d-97f0-7bf953f262f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.334044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70244f24-0a45-495d-97f0-7bf953f262f0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.334331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70244f24-0a45-495d-97f0-7bf953f262f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.334609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70244f24-0a45-495d-97f0-7bf953f262f0 None admin] PATH_INFO: `/v3/users/618682c43f9c4ab29adf65cff57f6aa3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.335825 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70244f24-0a45-495d-97f0-7bf953f262f0 None admin] RBAC: Authorizing `identity:get_user(user_id=618682c43f9c4ab29adf65cff57f6aa3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.337311 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70244f24-0a45-495d-97f0-7bf953f262f0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.338942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 145/291] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:45 2022] GET /identity/v3/users/618682c43f9c4ab29adf65cff57f6aa3 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:45.344891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fad2e-e649-4c06-8f22-765fa275ec6c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.461543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:45.487215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fad2e-e649-4c06-8f22-765fa275ec6c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.531706 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ab0d6-57a0-467c-b805-309ae3bd968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.532750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ab0d6-57a0-467c-b805-309ae3bd968f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.533254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ab0d6-57a0-467c-b805-309ae3bd968f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.533536 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ab0d6-57a0-467c-b805-309ae3bd968f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.604426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ab0d6-57a0-467c-b805-309ae3bd968f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.606223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ab0d6-57a0-467c-b805-309ae3bd968f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.686626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:45.723509 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 147/292] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:45.729202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70244f24-0a45-495d-97f0-7bf953f262f0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:45.856800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70244f24-0a45-495d-97f0-7bf953f262f0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:45.904021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80ac19-43d9-4ea8-a6d6-aa68d9cb05d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:45.905704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80ac19-43d9-4ea8-a6d6-aa68d9cb05d5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:45.906174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80ac19-43d9-4ea8-a6d6-aa68d9cb05d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:45.906623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80ac19-43d9-4ea8-a6d6-aa68d9cb05d5 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:45.913072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80ac19-43d9-4ea8-a6d6-aa68d9cb05d5 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:45.914671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80ac19-43d9-4ea8-a6d6-aa68d9cb05d5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:45.916879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 146/293] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:45 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:45.923059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ab0d6-57a0-467c-b805-309ae3bd968f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:46.046984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ab0d6-57a0-467c-b805-309ae3bd968f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:46.098673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b06fd4-0181-417e-82d1-bc522733a146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:46.099331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06fd4-0181-417e-82d1-bc522733a146 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:46.099616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06fd4-0181-417e-82d1-bc522733a146 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:46.099893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06fd4-0181-417e-82d1-bc522733a146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:46.100746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b06fd4-0181-417e-82d1-bc522733a146 None admin] RBAC: Adding query filter params (role.id=177939861d6e4e82b48401be184d66b4, scope.project.id=682d8892f2d3403f862be249dc3a8acf, user.id=618682c43f9c4ab29adf65cff57f6aa3) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:46.101058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b06fd4-0181-417e-82d1-bc522733a146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:46.102507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b06fd4-0181-417e-82d1-bc522733a146 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:46.109865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 148/294] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:45 2022] GET /identity/v3/role_assignments?user.id=618682c43f9c4ab29adf65cff57f6aa3&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=177939861d6e4e82b48401be184d66b4 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:46.936714 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af9928-c217-4c33-8646-5cfd8c4c0178 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:46.937223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af9928-c217-4c33-8646-5cfd8c4c0178 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:46.937298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af9928-c217-4c33-8646-5cfd8c4c0178 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:46.938148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8052735-2ada-4529-b4e8-1d2e63237b39 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:46.938630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 147/295] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:46.938775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8052735-2ada-4529-b4e8-1d2e63237b39 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:46.939127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8052735-2ada-4529-b4e8-1d2e63237b39 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:46.940728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 149/296] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:46.943735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947dc7-1dd4-4b77-8fde-82b2dd0ffafc None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:46.944263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947dc7-1dd4-4b77-8fde-82b2dd0ffafc None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:46.944798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947dc7-1dd4-4b77-8fde-82b2dd0ffafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:46.946138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0e48d-8f21-4dba-a24c-ff3ab088e23c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:46.946622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0e48d-8f21-4dba-a24c-ff3ab088e23c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:46.947091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0e48d-8f21-4dba-a24c-ff3ab088e23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:46.965049 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ca947dc7-1dd4-4b77-8fde-82b2dd0ffafc None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:46.965768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-08c0e48d-8f21-4dba-a24c-ff3ab088e23c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:46.966798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ca947dc7-1dd4-4b77-8fde-82b2dd0ffafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:46.967104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-08c0e48d-8f21-4dba-a24c-ff3ab088e23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:47.006597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:47.023387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:47.041767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 150/297] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:47.046574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 148/298] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:47.067037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0e48d-8f21-4dba-a24c-ff3ab088e23c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:47.070263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fba1c-6358-4167-836b-a6408796abb9 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.070562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fba1c-6358-4167-836b-a6408796abb9 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.070843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fba1c-6358-4167-836b-a6408796abb9 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.072376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 149/299] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:47.073017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fba1c-6358-4167-836b-a6408796abb9 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:47.173244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0e48d-8f21-4dba-a24c-ff3ab088e23c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:47.183148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fba1c-6358-4167-836b-a6408796abb9 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:47.208841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747c649-6b7b-43b1-b9e6-7ee08f1ca94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:47.209572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747c649-6b7b-43b1-b9e6-7ee08f1ca94b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.209851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747c649-6b7b-43b1-b9e6-7ee08f1ca94b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.210283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747c649-6b7b-43b1-b9e6-7ee08f1ca94b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.211587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747c649-6b7b-43b1-b9e6-7ee08f1ca94b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:47.213198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747c649-6b7b-43b1-b9e6-7ee08f1ca94b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:47.215248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 151/300] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:47.217282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0ca9d-3ff9-4bcb-9b3f-afc462c9f716 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.217573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0ca9d-3ff9-4bcb-9b3f-afc462c9f716 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.217853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0ca9d-3ff9-4bcb-9b3f-afc462c9f716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.221589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2c27d-87b1-462a-8d4e-fdd2c77fe6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:47.222390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2c27d-87b1-462a-8d4e-fdd2c77fe6c2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.222676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2c27d-87b1-462a-8d4e-fdd2c77fe6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.222956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2c27d-87b1-462a-8d4e-fdd2c77fe6c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.234349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2c27d-87b1-462a-8d4e-fdd2c77fe6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:47.235537 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d0a0ca9d-3ff9-4bcb-9b3f-afc462c9f716 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:47.236760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d0a0ca9d-3ff9-4bcb-9b3f-afc462c9f716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-10c2c27d-87b1-462a-8d4e-fdd2c77fe6c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:47.243990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:47.244902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 150/301] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:47.245694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2c27d-87b1-462a-8d4e-fdd2c77fe6c2 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:47.276500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:47.298356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:47.320429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 152/302] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:47.321346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0ca9d-3ff9-4bcb-9b3f-afc462c9f716 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:47.367422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2c27d-87b1-462a-8d4e-fdd2c77fe6c2 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:47.401797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b81073-58e2-45dd-b2cf-fd145e14b587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:47.402467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b81073-58e2-45dd-b2cf-fd145e14b587 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.402749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b81073-58e2-45dd-b2cf-fd145e14b587 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.403022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b81073-58e2-45dd-b2cf-fd145e14b587 None admin] PATH_INFO: `/v3/groups` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.403701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b81073-58e2-45dd-b2cf-fd145e14b587 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:47.405680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b81073-58e2-45dd-b2cf-fd145e14b587 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:47.436879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0ca9d-3ff9-4bcb-9b3f-afc462c9f716 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:47.470127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328698c-18c6-45f7-bd5e-5b7c03d123cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:47.470727 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328698c-18c6-45f7-bd5e-5b7c03d123cd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.471013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328698c-18c6-45f7-bd5e-5b7c03d123cd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.471290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328698c-18c6-45f7-bd5e-5b7c03d123cd None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.472023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328698c-18c6-45f7-bd5e-5b7c03d123cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:47.472344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328698c-18c6-45f7-bd5e-5b7c03d123cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:47.473708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328698c-18c6-45f7-bd5e-5b7c03d123cd None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:47.476733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 151/303] 10.222.0.23 () {60 vars in 1134 bytes} [Mon May 23 15:36:47 2022] POST /identity/v3/groups => generated 229 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:47.477377 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b81073-58e2-45dd-b2cf-fd145e14b587 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:47.481912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 153/304] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:47.483812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae04a15-1839-4b6b-8513-c8837bb97cca None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.484101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae04a15-1839-4b6b-8513-c8837bb97cca None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.484397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae04a15-1839-4b6b-8513-c8837bb97cca None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.485894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 154/305] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:47.488654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae04a15-1839-4b6b-8513-c8837bb97cca None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:47.523508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:47.603771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae04a15-1839-4b6b-8513-c8837bb97cca None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:47.608746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b81073-58e2-45dd-b2cf-fd145e14b587 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:47.610660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:47.643343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68c262-6d44-492d-b6a3-3161229b4d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:47.644227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c262-6d44-492d-b6a3-3161229b4d55 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.644551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c262-6d44-492d-b6a3-3161229b4d55 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.644840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c262-6d44-492d-b6a3-3161229b4d55 None admin] PATH_INFO: `/v3/users/416443d4ca2d432aa30277d60a00c9ba` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.647906 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81687b40-920e-4a23-97b7-3f1d1b21ab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:47.648750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81687b40-920e-4a23-97b7-3f1d1b21ab9a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.649035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81687b40-920e-4a23-97b7-3f1d1b21ab9a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.649307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81687b40-920e-4a23-97b7-3f1d1b21ab9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.661448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81687b40-920e-4a23-97b7-3f1d1b21ab9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:47.664289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68c262-6d44-492d-b6a3-3161229b4d55 None admin] RBAC: Authorizing `identity:get_user(user_id=416443d4ca2d432aa30277d60a00c9ba)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:47.664289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68c262-6d44-492d-b6a3-3161229b4d55 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:47.669898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 155/306] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:47 2022] GET /identity/v3/users/416443d4ca2d432aa30277d60a00c9ba => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:47.669898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4c01b-13c7-49ef-834f-a24b5c06c192 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.669898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4c01b-13c7-49ef-834f-a24b5c06c192 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.669898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4c01b-13c7-49ef-834f-a24b5c06c192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-81687b40-920e-4a23-97b7-3f1d1b21ab9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:47.670177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:47.670985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:47.672216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 152/307] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:47.673244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81687b40-920e-4a23-97b7-3f1d1b21ab9a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:47.691501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-27d4c01b-13c7-49ef-834f-a24b5c06c192 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:47.693318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-27d4c01b-13c7-49ef-834f-a24b5c06c192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:47.767426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 156/308] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:47.768657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4c01b-13c7-49ef-834f-a24b5c06c192 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:47.785519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81687b40-920e-4a23-97b7-3f1d1b21ab9a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:47.797064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:47.820885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b85e7-02f4-464b-81fb-c8dd0cb8b5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:47.821551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b85e7-02f4-464b-81fb-c8dd0cb8b5a4 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.821833 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b85e7-02f4-464b-81fb-c8dd0cb8b5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.822111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b85e7-02f4-464b-81fb-c8dd0cb8b5a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.823035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b85e7-02f4-464b-81fb-c8dd0cb8b5a4 None admin] RBAC: Adding query filter params (role.id=2a5bf966af5a486fa21c10760ebfbcad, user.id=416443d4ca2d432aa30277d60a00c9ba, scope.system=all) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:47.823260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b85e7-02f4-464b-81fb-c8dd0cb8b5a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:47.824723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b85e7-02f4-464b-81fb-c8dd0cb8b5a4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:47.840405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 153/309] 10.222.0.23 () {56 vars in 1308 bytes} [Mon May 23 15:36:47 2022] GET /identity/v3/role_assignments?user.id=416443d4ca2d432aa30277d60a00c9ba&scope.system=all&role.id=2a5bf966af5a486fa21c10760ebfbcad => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:47.840405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b85e7-02f4-464b-81fb-c8dd0cb8b5a4 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:47.880429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:47.901596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4c01b-13c7-49ef-834f-a24b5c06c192 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:47.955437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f661c-39c2-46d8-8cbf-965497ecc4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:47.959771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f661c-39c2-46d8-8cbf-965497ecc4ff None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:47.960409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f661c-39c2-46d8-8cbf-965497ecc4ff None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:47.961162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f661c-39c2-46d8-8cbf-965497ecc4ff None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:47.962432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f661c-39c2-46d8-8cbf-965497ecc4ff None admin] RBAC: Adding query filter params (name=member) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:47.963425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f661c-39c2-46d8-8cbf-965497ecc4ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:47.965659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f661c-39c2-46d8-8cbf-965497ecc4ff None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:47.972192 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b85e7-02f4-464b-81fb-c8dd0cb8b5a4 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:47.979512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 157/310] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:47.985470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f661c-39c2-46d8-8cbf-965497ecc4ff None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:48.020843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd047e28-6382-4b09-b1de-87ff6be2bbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:48.022607 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd047e28-6382-4b09-b1de-87ff6be2bbe0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:48.023370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd047e28-6382-4b09-b1de-87ff6be2bbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:48.023850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd047e28-6382-4b09-b1de-87ff6be2bbe0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:48.041928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd047e28-6382-4b09-b1de-87ff6be2bbe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-bd047e28-6382-4b09-b1de-87ff6be2bbe0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:48.052535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:48.053419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:48.054188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:48.056738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 154/311] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:48.062753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd047e28-6382-4b09-b1de-87ff6be2bbe0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:48.103519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f661c-39c2-46d8-8cbf-965497ecc4ff None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:48.108764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:48.144768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d9082-6019-404d-91c5-968daada0b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:48.145800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d9082-6019-404d-91c5-968daada0b57 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:48.146156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d9082-6019-404d-91c5-968daada0b57 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:48.146439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d9082-6019-404d-91c5-968daada0b57 None admin] PATH_INFO: `/v3/users/50f37600e12e469aa43fdb6e3ac483b3` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:48.148190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d9082-6019-404d-91c5-968daada0b57 None admin] RBAC: Authorizing `identity:get_user(user_id=50f37600e12e469aa43fdb6e3ac483b3)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:48.149676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d9082-6019-404d-91c5-968daada0b57 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:48.156503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 158/312] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:47 2022] GET /identity/v3/users/50f37600e12e469aa43fdb6e3ac483b3 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:48.157867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d9082-6019-404d-91c5-968daada0b57 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:48.175831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:48.191100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd047e28-6382-4b09-b1de-87ff6be2bbe0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:48.225883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7966f-d549-4a44-8318-c1284d4ad9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:48.226562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7966f-d549-4a44-8318-c1284d4ad9e0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:48.226848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7966f-d549-4a44-8318-c1284d4ad9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:48.227128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7966f-d549-4a44-8318-c1284d4ad9e0 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:48.227908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7966f-d549-4a44-8318-c1284d4ad9e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:48.228196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7966f-d549-4a44-8318-c1284d4ad9e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:48.229616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7966f-d549-4a44-8318-c1284d4ad9e0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:48.243913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 155/313] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:48.245593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7966f-d549-4a44-8318-c1284d4ad9e0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:48.279805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d9082-6019-404d-91c5-968daada0b57 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:48.321766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d8795-44d3-4212-a5bc-7945e623d6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:48.322889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d8795-44d3-4212-a5bc-7945e623d6e6 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:48.323240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d8795-44d3-4212-a5bc-7945e623d6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:48.323525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d8795-44d3-4212-a5bc-7945e623d6e6 None admin] PATH_INFO: `/v3/system/users/50f37600e12e469aa43fdb6e3ac483b3/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:48.325274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d8795-44d3-4212-a5bc-7945e623d6e6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=50f37600e12e469aa43fdb6e3ac483b3, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:48.327937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d8795-44d3-4212-a5bc-7945e623d6e6 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:48.371741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7966f-d549-4a44-8318-c1284d4ad9e0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:48.397020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 159/314] 10.222.0.23 () {58 vars in 1334 bytes} [Mon May 23 15:36:48 2022] PUT /identity/v3/system/users/50f37600e12e469aa43fdb6e3ac483b3/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:36:48.417442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80be6bd-e245-4464-9b68-2a1d5b446d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:48.418810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80be6bd-e245-4464-9b68-2a1d5b446d72 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:48.419295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80be6bd-e245-4464-9b68-2a1d5b446d72 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:48.419768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80be6bd-e245-4464-9b68-2a1d5b446d72 None admin] PATH_INFO: `/v3/users/156cca94b61c4cf18b45104afc32a9df` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:48.421646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80be6bd-e245-4464-9b68-2a1d5b446d72 None admin] RBAC: Authorizing `identity:get_user(user_id=156cca94b61c4cf18b45104afc32a9df)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:48.423814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80be6bd-e245-4464-9b68-2a1d5b446d72 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:48.426418 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 156/315] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:48 2022] GET /identity/v3/users/156cca94b61c4cf18b45104afc32a9df => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:48.432720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d8795-44d3-4212-a5bc-7945e623d6e6 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:48.451032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:48.551138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d8795-44d3-4212-a5bc-7945e623d6e6 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:48.589535 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0fe8b-990a-46fc-ad20-97050f7b7eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:48.590586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0fe8b-990a-46fc-ad20-97050f7b7eb4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:48.591065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0fe8b-990a-46fc-ad20-97050f7b7eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:48.591535 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0fe8b-990a-46fc-ad20-97050f7b7eb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:48.674552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0fe8b-990a-46fc-ad20-97050f7b7eb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:48.676392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0fe8b-990a-46fc-ad20-97050f7b7eb4 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:48.691523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:48.783817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 160/316] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:48.789743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80be6bd-e245-4464-9b68-2a1d5b446d72 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:48.809526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:48.894340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80be6bd-e245-4464-9b68-2a1d5b446d72 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:48.934499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8802b0-a0c5-4d0c-ae2b-bd787a181e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:48.935358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8802b0-a0c5-4d0c-ae2b-bd787a181e67 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:48.935673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8802b0-a0c5-4d0c-ae2b-bd787a181e67 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:48.935942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8802b0-a0c5-4d0c-ae2b-bd787a181e67 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:48.937157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8802b0-a0c5-4d0c-ae2b-bd787a181e67 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:48.938579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8802b0-a0c5-4d0c-ae2b-bd787a181e67 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:48.940452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 157/317] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:48 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:48.946266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0fe8b-990a-46fc-ad20-97050f7b7eb4 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:49.050632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0fe8b-990a-46fc-ad20-97050f7b7eb4 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:49.063558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:49.084784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f6849-c4a8-4f43-8134-196624757f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:49.085451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f6849-c4a8-4f43-8134-196624757f95 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:49.085739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f6849-c4a8-4f43-8134-196624757f95 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:49.086019 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f6849-c4a8-4f43-8134-196624757f95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:49.086854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f6849-c4a8-4f43-8134-196624757f95 None admin] RBAC: Adding query filter params (role.id=2a5bf966af5a486fa21c10760ebfbcad, scope.project.id=682d8892f2d3403f862be249dc3a8acf, user.id=156cca94b61c4cf18b45104afc32a9df) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:49.087170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f6849-c4a8-4f43-8134-196624757f95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:49.088640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f6849-c4a8-4f43-8134-196624757f95 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:49.095511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 161/318] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:36:48 2022] GET /identity/v3/role_assignments?user.id=156cca94b61c4cf18b45104afc32a9df&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=2a5bf966af5a486fa21c10760ebfbcad => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:49.754451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a420a-1069-4e5a-94ab-7a5e3aea3ec6 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:49.754959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a420a-1069-4e5a-94ab-7a5e3aea3ec6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:49.755032 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a420a-1069-4e5a-94ab-7a5e3aea3ec6 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:49.756422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 158/319] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:49.761868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33efe14-e67c-4e1f-9350-44b47aab360b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:49.762249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33efe14-e67c-4e1f-9350-44b47aab360b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:49.762441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33efe14-e67c-4e1f-9350-44b47aab360b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:49.779538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b33efe14-e67c-4e1f-9350-44b47aab360b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:49.780765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b33efe14-e67c-4e1f-9350-44b47aab360b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:49.848690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 162/320] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:49.872491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a420a-1069-4e5a-94ab-7a5e3aea3ec6 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:49.960637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:49.977030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a420a-1069-4e5a-94ab-7a5e3aea3ec6 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:50.012170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1057dd-040d-4e17-931f-ef740e1317c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:50.012920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1057dd-040d-4e17-931f-ef740e1317c5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.013207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1057dd-040d-4e17-931f-ef740e1317c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.013490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1057dd-040d-4e17-931f-ef740e1317c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.014700 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1057dd-040d-4e17-931f-ef740e1317c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:50.016139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1057dd-040d-4e17-931f-ef740e1317c5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:50.017947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 159/321] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:36:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:50.023446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33efe14-e67c-4e1f-9350-44b47aab360b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:50.060454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd0cfa-39d6-41e2-b72c-23911d714e81 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.060812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd0cfa-39d6-41e2-b72c-23911d714e81 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.061108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd0cfa-39d6-41e2-b72c-23911d714e81 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.063074 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 160/322] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:50.067810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940975f-4835-4bc9-9828-443f3d2fec7b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.068100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940975f-4835-4bc9-9828-443f3d2fec7b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.068397 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940975f-4835-4bc9-9828-443f3d2fec7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.084724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-5940975f-4835-4bc9-9828-443f3d2fec7b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:50.085913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5940975f-4835-4bc9-9828-443f3d2fec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:50.126296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33efe14-e67c-4e1f-9350-44b47aab360b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:50.134557 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:50.154089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 161/323] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:50.162475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c96f1-58c5-4771-ba85-afd2a1cafd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:50.163153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c96f1-58c5-4771-ba85-afd2a1cafd93 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.163472 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c96f1-58c5-4771-ba85-afd2a1cafd93 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.163708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c96f1-58c5-4771-ba85-afd2a1cafd93 None admin] PATH_INFO: `/v3/groups` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.164433 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c96f1-58c5-4771-ba85-afd2a1cafd93 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:50.165873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c96f1-58c5-4771-ba85-afd2a1cafd93 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:50.177087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940975f-4835-4bc9-9828-443f3d2fec7b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:50.255622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:50.265237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 163/324] 10.222.0.23 () {60 vars in 1134 bytes} [Mon May 23 15:36:50 2022] POST /identity/v3/groups => generated 226 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:36:50.275803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940975f-4835-4bc9-9828-443f3d2fec7b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:50.317162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6efe30-e202-409f-b368-d88de961cbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:50.317972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6efe30-e202-409f-b368-d88de961cbae None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.318252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6efe30-e202-409f-b368-d88de961cbae None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.318524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6efe30-e202-409f-b368-d88de961cbae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.330589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6efe30-e202-409f-b368-d88de961cbae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-bc6efe30-e202-409f-b368-d88de961cbae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:50.338547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:50.339469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:50.340490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 162/325] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:50.348496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c96f1-58c5-4771-ba85-afd2a1cafd93 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:50.460214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c96f1-58c5-4771-ba85-afd2a1cafd93 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:50.494015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4504e-d8ec-4ca3-9e33-3ba173f8ae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:50.494692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4504e-d8ec-4ca3-9e33-3ba173f8ae00 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.494982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4504e-d8ec-4ca3-9e33-3ba173f8ae00 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.495273 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4504e-d8ec-4ca3-9e33-3ba173f8ae00 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.496025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4504e-d8ec-4ca3-9e33-3ba173f8ae00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:50.496340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4504e-d8ec-4ca3-9e33-3ba173f8ae00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:50.497759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4504e-d8ec-4ca3-9e33-3ba173f8ae00 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:50.506909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 164/326] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:50.512897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6efe30-e202-409f-b368-d88de961cbae None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:50.631931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6efe30-e202-409f-b368-d88de961cbae None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:50.668515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f655cb3-d23a-4da5-ac62-e0fbf8e86b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:50.669389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f655cb3-d23a-4da5-ac62-e0fbf8e86b65 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.669668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f655cb3-d23a-4da5-ac62-e0fbf8e86b65 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.670492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f655cb3-d23a-4da5-ac62-e0fbf8e86b65 None admin] PATH_INFO: `/v3/users/416443d4ca2d432aa30277d60a00c9ba` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.673141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f655cb3-d23a-4da5-ac62-e0fbf8e86b65 None admin] RBAC: Authorizing `identity:get_user(user_id=416443d4ca2d432aa30277d60a00c9ba)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:50.674816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f655cb3-d23a-4da5-ac62-e0fbf8e86b65 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:50.677569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 163/327] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:50 2022] GET /identity/v3/users/416443d4ca2d432aa30277d60a00c9ba => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:50.682722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4504e-d8ec-4ca3-9e33-3ba173f8ae00 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:50.718815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:50.725909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe328c44-7edc-43a8-906f-2ad2b0acf474 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.726201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe328c44-7edc-43a8-906f-2ad2b0acf474 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.726476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe328c44-7edc-43a8-906f-2ad2b0acf474 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.729466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 164/328] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:50.733607 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ba325-9527-43af-a350-fa2112f498dd None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.734078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ba325-9527-43af-a350-fa2112f498dd None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.734179 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ba325-9527-43af-a350-fa2112f498dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.754245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8a7ba325-9527-43af-a350-fa2112f498dd None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:50.755557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8a7ba325-9527-43af-a350-fa2112f498dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:50.758608 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:50.821277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4504e-d8ec-4ca3-9e33-3ba173f8ae00 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:50.839668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 165/329] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:50.860578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5785a07-6b0b-4775-a3e7-9e1e1cd00771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:50.861938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785a07-6b0b-4775-a3e7-9e1e1cd00771 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:50.862436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785a07-6b0b-4775-a3e7-9e1e1cd00771 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:50.862919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785a07-6b0b-4775-a3e7-9e1e1cd00771 None admin] PATH_INFO: `/v3/system/users/416443d4ca2d432aa30277d60a00c9ba/roles/2a5bf966af5a486fa21c10760ebfbcad` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:50.864753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5785a07-6b0b-4775-a3e7-9e1e1cd00771 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=416443d4ca2d432aa30277d60a00c9ba, role_id=2a5bf966af5a486fa21c10760ebfbcad)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:50.867239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ba325-9527-43af-a350-fa2112f498dd None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:50.872688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5785a07-6b0b-4775-a3e7-9e1e1cd00771 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:50.947296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 165/330] 10.222.0.23 () {58 vars in 1334 bytes} [Mon May 23 15:36:50 2022] PUT /identity/v3/system/users/416443d4ca2d432aa30277d60a00c9ba/roles/2a5bf966af5a486fa21c10760ebfbcad => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:36:50.989519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ba325-9527-43af-a350-fa2112f498dd None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:51.029789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f5937-9d7d-4b97-aec2-faa735597a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:51.030966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f5937-9d7d-4b97-aec2-faa735597a93 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:51.031449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f5937-9d7d-4b97-aec2-faa735597a93 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:51.031920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f5937-9d7d-4b97-aec2-faa735597a93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:51.041521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:51.046056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f5937-9d7d-4b97-aec2-faa735597a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-780f5937-9d7d-4b97-aec2-faa735597a93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:51.054221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:51.061448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:51.062240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 166/331] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:51.062936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5785a07-6b0b-4775-a3e7-9e1e1cd00771 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:51.178010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:51.189621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5785a07-6b0b-4775-a3e7-9e1e1cd00771 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:51.229248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636c572-0842-4e4a-b70c-f129c659d341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:51.229514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636c572-0842-4e4a-b70c-f129c659d341 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:51.229775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636c572-0842-4e4a-b70c-f129c659d341 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:51.230054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636c572-0842-4e4a-b70c-f129c659d341 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:51.230816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636c572-0842-4e4a-b70c-f129c659d341 None admin] RBAC: Adding query filter params (name=member) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:51.231126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636c572-0842-4e4a-b70c-f129c659d341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:51.233089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636c572-0842-4e4a-b70c-f129c659d341 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:51.242212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 166/332] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:51.247896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f5937-9d7d-4b97-aec2-faa735597a93 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:51.374441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f5937-9d7d-4b97-aec2-faa735597a93 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:51.410399 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40ed79-23d6-46d7-8a92-5eb65f223740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:51.411348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40ed79-23d6-46d7-8a92-5eb65f223740 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:51.411632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40ed79-23d6-46d7-8a92-5eb65f223740 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:51.411911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40ed79-23d6-46d7-8a92-5eb65f223740 None admin] PATH_INFO: `/v3/users/50f37600e12e469aa43fdb6e3ac483b3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:51.413199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40ed79-23d6-46d7-8a92-5eb65f223740 None admin] RBAC: Authorizing `identity:get_user(user_id=50f37600e12e469aa43fdb6e3ac483b3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:51.414695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40ed79-23d6-46d7-8a92-5eb65f223740 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:51.416581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 167/333] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:51 2022] GET /identity/v3/users/50f37600e12e469aa43fdb6e3ac483b3 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:51.422494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636c572-0842-4e4a-b70c-f129c659d341 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:51.542337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636c572-0842-4e4a-b70c-f129c659d341 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:51.584918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93240b-cbcf-46ab-9e5a-9b26c891cff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:51.585821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93240b-cbcf-46ab-9e5a-9b26c891cff7 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:51.586250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93240b-cbcf-46ab-9e5a-9b26c891cff7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:51.586694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93240b-cbcf-46ab-9e5a-9b26c891cff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:51.587672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93240b-cbcf-46ab-9e5a-9b26c891cff7 None admin] RBAC: Adding query filter params (role.id=177939861d6e4e82b48401be184d66b4, user.id=50f37600e12e469aa43fdb6e3ac483b3, scope.system=all) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:51.588140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93240b-cbcf-46ab-9e5a-9b26c891cff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:51.589762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93240b-cbcf-46ab-9e5a-9b26c891cff7 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:51.607632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 167/334] 10.222.0.23 () {56 vars in 1308 bytes} [Mon May 23 15:36:51 2022] GET /identity/v3/role_assignments?user.id=50f37600e12e469aa43fdb6e3ac483b3&scope.system=all&role.id=177939861d6e4e82b48401be184d66b4 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:53.206157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d655150-e6d0-4d8c-8bd7-ad9ea73ad241 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:53.207196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d655150-e6d0-4d8c-8bd7-ad9ea73ad241 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:53.207683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d655150-e6d0-4d8c-8bd7-ad9ea73ad241 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:53.209632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 168/335] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:53.219616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f0d25-4b1b-4880-ad86-00f476e6ea4b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:53.220440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f0d25-4b1b-4880-ad86-00f476e6ea4b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:53.220942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f0d25-4b1b-4880-ad86-00f476e6ea4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:53.240347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-714f0d25-4b1b-4880-ad86-00f476e6ea4b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:53.241845 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-714f0d25-4b1b-4880-ad86-00f476e6ea4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:53.250775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:53.326948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 168/336] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:53.381509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d655150-e6d0-4d8c-8bd7-ad9ea73ad241 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:53.468914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:53.512920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d655150-e6d0-4d8c-8bd7-ad9ea73ad241 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:53.551950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2b325-fce5-49e7-9bfd-ecf94f61ecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:53.552783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2b325-fce5-49e7-9bfd-ecf94f61ecfe None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:53.553065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2b325-fce5-49e7-9bfd-ecf94f61ecfe None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:53.553339 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2b325-fce5-49e7-9bfd-ecf94f61ecfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:53.567816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2b325-fce5-49e7-9bfd-ecf94f61ecfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-3fa2b325-fce5-49e7-9bfd-ecf94f61ecfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:53.575570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:53.577050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:53.577850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 169/337] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:53.585932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f0d25-4b1b-4880-ad86-00f476e6ea4b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:53.675236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca37a57-8f7a-4ef1-89da-2eddde300f5b None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:53.675893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca37a57-8f7a-4ef1-89da-2eddde300f5b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:53.676410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca37a57-8f7a-4ef1-89da-2eddde300f5b None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:53.678000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 170/338] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:53.685525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf4f5b-8846-445a-a29a-6cdabb385445 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:53.686112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf4f5b-8846-445a-a29a-6cdabb385445 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:53.686592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf4f5b-8846-445a-a29a-6cdabb385445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:53.717955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a6cf4f5b-8846-445a-a29a-6cdabb385445 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:53.719572 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a6cf4f5b-8846-445a-a29a-6cdabb385445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:53.745042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f0d25-4b1b-4880-ad86-00f476e6ea4b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:53.814883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 171/339] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:53.819316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01d164-4311-4b5b-bd5b-51fd49ac4cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:53.819997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01d164-4311-4b5b-bd5b-51fd49ac4cb0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:53.820283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01d164-4311-4b5b-bd5b-51fd49ac4cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:53.820590 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01d164-4311-4b5b-bd5b-51fd49ac4cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:53.821352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01d164-4311-4b5b-bd5b-51fd49ac4cb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:53.821662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01d164-4311-4b5b-bd5b-51fd49ac4cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:53.823034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01d164-4311-4b5b-bd5b-51fd49ac4cb0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:53.836459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 169/340] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:53.845138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf4f5b-8846-445a-a29a-6cdabb385445 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:53.861442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01d164-4311-4b5b-bd5b-51fd49ac4cb0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:53.868809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:53.974017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.082355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf4f5b-8846-445a-a29a-6cdabb385445 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.099675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01d164-4311-4b5b-bd5b-51fd49ac4cb0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.166400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bdce6-e61e-4ba1-be32-3daf0319c2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.167431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bdce6-e61e-4ba1-be32-3daf0319c2e2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.168000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bdce6-e61e-4ba1-be32-3daf0319c2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.168499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bdce6-e61e-4ba1-be32-3daf0319c2e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.193450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a2760-a073-4210-a8db-ffb1f9b5e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.195174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a2760-a073-4210-a8db-ffb1f9b5e5ed None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.195886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a2760-a073-4210-a8db-ffb1f9b5e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.196636 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a2760-a073-4210-a8db-ffb1f9b5e5ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.222028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a2760-a073-4210-a8db-ffb1f9b5e5ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ff0a2760-a073-4210-a8db-ffb1f9b5e5ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:54.248822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:54.249745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:54.250538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:54.251153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 170/341] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:36:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:54.259828 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a2760-a073-4210-a8db-ffb1f9b5e5ed None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.341228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bdce6-e61e-4ba1-be32-3daf0319c2e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.343438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bdce6-e61e-4ba1-be32-3daf0319c2e2 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.392916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.486817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a2760-a073-4210-a8db-ffb1f9b5e5ed None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.493919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.533070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab712a80-593c-4b2f-b703-0c063bfb0fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.534231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab712a80-593c-4b2f-b703-0c063bfb0fa5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.535037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab712a80-593c-4b2f-b703-0c063bfb0fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.535789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab712a80-593c-4b2f-b703-0c063bfb0fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.536929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab712a80-593c-4b2f-b703-0c063bfb0fa5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:54.537740 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab712a80-593c-4b2f-b703-0c063bfb0fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.540264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab712a80-593c-4b2f-b703-0c063bfb0fa5 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.540889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 172/342] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:36:54.543496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225fe8d-814e-4e02-bb20-7297181befe6 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.543625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225fe8d-814e-4e02-bb20-7297181befe6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.548382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225fe8d-814e-4e02-bb20-7297181befe6 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.548382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 173/343] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:54.548382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee7dc3-f7c5-4813-b1a6-23f6f56611f9 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.548382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee7dc3-f7c5-4813-b1a6-23f6f56611f9 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.548382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee7dc3-f7c5-4813-b1a6-23f6f56611f9 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.553051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 174/344] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:54.554353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600940cd-9041-470e-803b-c05ae611305f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.554639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600940cd-9041-470e-803b-c05ae611305f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.554920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600940cd-9041-470e-803b-c05ae611305f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.556463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 171/345] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:36:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:54.558466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb78a56-62ee-487e-aacd-53601a9ad1d0 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.558977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb78a56-62ee-487e-aacd-53601a9ad1d0 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.559416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb78a56-62ee-487e-aacd-53601a9ad1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.574216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-600940cd-9041-470e-803b-c05ae611305f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:54.575387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-600940cd-9041-470e-803b-c05ae611305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.580452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-bdb78a56-62ee-487e-aacd-53601a9ad1d0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:54.581805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bdb78a56-62ee-487e-aacd-53601a9ad1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:54.648147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 175/346] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:54.649103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600940cd-9041-470e-803b-c05ae611305f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.671233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 172/347] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:54.672302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb78a56-62ee-487e-aacd-53601a9ad1d0 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.751425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.765492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.778416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600940cd-9041-470e-803b-c05ae611305f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.799244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb78a56-62ee-487e-aacd-53601a9ad1d0 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.814431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7714b57-5153-4afa-b360-1f583c104842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.820786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7714b57-5153-4afa-b360-1f583c104842 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.820786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7714b57-5153-4afa-b360-1f583c104842 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.820786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7714b57-5153-4afa-b360-1f583c104842 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.820786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7714b57-5153-4afa-b360-1f583c104842 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.820786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7714b57-5153-4afa-b360-1f583c104842 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.820786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 176/348] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:54 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:54.821262 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7714b57-5153-4afa-b360-1f583c104842 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:54.840597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3f31c-fdb2-493b-9c3a-ca2b287444e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.841280 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3f31c-fdb2-493b-9c3a-ca2b287444e7 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.841573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3f31c-fdb2-493b-9c3a-ca2b287444e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.842383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3f31c-fdb2-493b-9c3a-ca2b287444e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.917432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3f31c-fdb2-493b-9c3a-ca2b287444e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:54.918902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3f31c-fdb2-493b-9c3a-ca2b287444e7 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:54.929223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7714b57-5153-4afa-b360-1f583c104842 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:54.964190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee0649-6bcb-4306-b43c-63b08ef87442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:54.965047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee0649-6bcb-4306-b43c-63b08ef87442 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:54.965325 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee0649-6bcb-4306-b43c-63b08ef87442 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:54.965594 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee0649-6bcb-4306-b43c-63b08ef87442 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:54.977916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee0649-6bcb-4306-b43c-63b08ef87442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:54.981829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-dfee0649-6bcb-4306-b43c-63b08ef87442 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:54.986188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:54.987067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:54.987866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:54.992149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 177/349] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:54.998222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee0649-6bcb-4306-b43c-63b08ef87442 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.012448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.026691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 173/350] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:55.027552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3f31c-fdb2-493b-9c3a-ca2b287444e7 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.104504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee0649-6bcb-4306-b43c-63b08ef87442 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.140302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3f31c-fdb2-493b-9c3a-ca2b287444e7 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.155264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c856988-c8ed-40fb-bf4a-895f6ac17d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.155264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c856988-c8ed-40fb-bf4a-895f6ac17d83 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.155264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c856988-c8ed-40fb-bf4a-895f6ac17d83 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.155264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c856988-c8ed-40fb-bf4a-895f6ac17d83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.158783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c856988-c8ed-40fb-bf4a-895f6ac17d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-9c856988-c8ed-40fb-bf4a-895f6ac17d83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:55.166839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:55.167737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:55.168557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 178/351] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:55.172690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c856988-c8ed-40fb-bf4a-895f6ac17d83 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.186351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09879d73-a0f0-4c3f-a0c3-1c55de123ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.187767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879d73-a0f0-4c3f-a0c3-1c55de123ca5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.188060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879d73-a0f0-4c3f-a0c3-1c55de123ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.188437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879d73-a0f0-4c3f-a0c3-1c55de123ca5 None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.197089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09879d73-a0f0-4c3f-a0c3-1c55de123ca5 None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.199128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09879d73-a0f0-4c3f-a0c3-1c55de123ca5 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.202479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 174/352] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:55.203983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09879d73-a0f0-4c3f-a0c3-1c55de123ca5 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.233497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.272838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.310995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c856988-c8ed-40fb-bf4a-895f6ac17d83 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.350932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09879d73-a0f0-4c3f-a0c3-1c55de123ca5 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.360141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a7edf-e9ab-4fef-8b9c-d4f03c8c0807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.360777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a7edf-e9ab-4fef-8b9c-d4f03c8c0807 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.361056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a7edf-e9ab-4fef-8b9c-d4f03c8c0807 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.361798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a7edf-e9ab-4fef-8b9c-d4f03c8c0807 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.362566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a7edf-e9ab-4fef-8b9c-d4f03c8c0807 None admin] RBAC: Adding query filter params (name=member) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:55.363284 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a7edf-e9ab-4fef-8b9c-d4f03c8c0807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.364983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a7edf-e9ab-4fef-8b9c-d4f03c8c0807 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.376949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 179/353] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:55.381319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a7edf-e9ab-4fef-8b9c-d4f03c8c0807 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.390861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e49e14-708f-4b81-886f-eb6b44824cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.391721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e49e14-708f-4b81-886f-eb6b44824cbf None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.392005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e49e14-708f-4b81-886f-eb6b44824cbf None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.392279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e49e14-708f-4b81-886f-eb6b44824cbf None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.394074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e49e14-708f-4b81-886f-eb6b44824cbf None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.395572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e49e14-708f-4b81-886f-eb6b44824cbf None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.397433 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 175/354] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:55.398038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e49e14-708f-4b81-886f-eb6b44824cbf None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.517601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.517784 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a7edf-e9ab-4fef-8b9c-d4f03c8c0807 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.530186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e49e14-708f-4b81-886f-eb6b44824cbf None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.562126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7a259-a422-4bd4-862d-34fd31335467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.562947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7a259-a422-4bd4-862d-34fd31335467 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.563299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7a259-a422-4bd4-862d-34fd31335467 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.563653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7a259-a422-4bd4-862d-34fd31335467 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.564661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7a259-a422-4bd4-862d-34fd31335467 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:55.565046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7a259-a422-4bd4-862d-34fd31335467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.568598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7a259-a422-4bd4-862d-34fd31335467 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.575547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11e5fd-97a8-44c9-a8f4-3c8e03643902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.575739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.576239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11e5fd-97a8-44c9-a8f4-3c8e03643902 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.577108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11e5fd-97a8-44c9-a8f4-3c8e03643902 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.577440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11e5fd-97a8-44c9-a8f4-3c8e03643902 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.578313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11e5fd-97a8-44c9-a8f4-3c8e03643902 None admin] RBAC: Adding query filter params (group.id=c61b7da72b3643489b6cb3990133a99a, role.id=177939861d6e4e82b48401be184d66b4, scope.project.id=9811a47b7d5449f999b0f971ee26f008) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:55.578625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11e5fd-97a8-44c9-a8f4-3c8e03643902 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.584743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 180/355] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:55.584743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5e5b3-6f68-421b-822f-3684cbabdeb3 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.584743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5e5b3-6f68-421b-822f-3684cbabdeb3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.584743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5e5b3-6f68-421b-822f-3684cbabdeb3 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.584743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 181/356] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:55.584743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5e5b3-6f68-421b-822f-3684cbabdeb3 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.585190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11e5fd-97a8-44c9-a8f4-3c8e03643902 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.593929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 176/357] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/role_assignments?group.id=c61b7da72b3643489b6cb3990133a99a&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=177939861d6e4e82b48401be184d66b4 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:55.595995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11e5fd-97a8-44c9-a8f4-3c8e03643902 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.725123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5e5b3-6f68-421b-822f-3684cbabdeb3 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.733948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11e5fd-97a8-44c9-a8f4-3c8e03643902 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.783899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85869a2-bab3-45c7-b22d-05308d0a1a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.785372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85869a2-bab3-45c7-b22d-05308d0a1a15 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.785891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85869a2-bab3-45c7-b22d-05308d0a1a15 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.786298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85869a2-bab3-45c7-b22d-05308d0a1a15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.789546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca72d2-f8ad-461f-b05d-8e280970cce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.790560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca72d2-f8ad-461f-b05d-8e280970cce2 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.790948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca72d2-f8ad-461f-b05d-8e280970cce2 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.791328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca72d2-f8ad-461f-b05d-8e280970cce2 None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.793191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca72d2-f8ad-461f-b05d-8e280970cce2 None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.794760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca72d2-f8ad-461f-b05d-8e280970cce2 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.797426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 177/358] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:55.798417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca72d2-f8ad-461f-b05d-8e280970cce2 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:55.835335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.886487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:55.890393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85869a2-bab3-45c7-b22d-05308d0a1a15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.892618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85869a2-bab3-45c7-b22d-05308d0a1a15 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.941156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca72d2-f8ad-461f-b05d-8e280970cce2 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:55.980773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2461223-123c-44b4-9acc-f03ad54b462b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:55.981881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2461223-123c-44b4-9acc-f03ad54b462b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.983489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2461223-123c-44b4-9acc-f03ad54b462b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.983920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2461223-123c-44b4-9acc-f03ad54b462b None admin] PATH_INFO: `/v3/users/416443d4ca2d432aa30277d60a00c9ba` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:55.985372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2461223-123c-44b4-9acc-f03ad54b462b None admin] RBAC: Authorizing `identity:get_user(user_id=416443d4ca2d432aa30277d60a00c9ba)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:55.988301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2461223-123c-44b4-9acc-f03ad54b462b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:55.988589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 178/359] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/users/416443d4ca2d432aa30277d60a00c9ba => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:55.994453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d26792-2893-42ed-8d8a-9c9a7c73a3f7 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:55.994787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d26792-2893-42ed-8d8a-9c9a7c73a3f7 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:55.995019 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d26792-2893-42ed-8d8a-9c9a7c73a3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.017691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 182/360] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:56.018127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-12d26792-2893-42ed-8d8a-9c9a7c73a3f7 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:56.018535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85869a2-bab3-45c7-b22d-05308d0a1a15 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.019417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-12d26792-2893-42ed-8d8a-9c9a7c73a3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:56.105766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 179/361] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:56.112515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c58e26-4b9b-46ce-8e91-ee4df5e4982c None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.112515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c58e26-4b9b-46ce-8e91-ee4df5e4982c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.112515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c58e26-4b9b-46ce-8e91-ee4df5e4982c None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.112515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 180/362] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:56.112515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c58e26-4b9b-46ce-8e91-ee4df5e4982c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.138704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:56.148465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:56.151717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85869a2-bab3-45c7-b22d-05308d0a1a15 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:56.191428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fc8f0-c092-499c-9908-d6350e1cb99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:56.192094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fc8f0-c092-499c-9908-d6350e1cb99c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.196406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fc8f0-c092-499c-9908-d6350e1cb99c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.196406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fc8f0-c092-499c-9908-d6350e1cb99c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.196406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fc8f0-c092-499c-9908-d6350e1cb99c None admin] RBAC: Adding query filter params (group.id=c61b7da72b3643489b6cb3990133a99a, role.id=f7e0a506c7894b20b2b65fedfd2244e5, scope.project.id=9811a47b7d5449f999b0f971ee26f008) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:56.196406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fc8f0-c092-499c-9908-d6350e1cb99c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:56.196406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fc8f0-c092-499c-9908-d6350e1cb99c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:56.201465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 183/363] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 23 15:36:56 2022] GET /identity/v3/role_assignments?group.id=c61b7da72b3643489b6cb3990133a99a&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=f7e0a506c7894b20b2b65fedfd2244e5 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:56.202200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fc8f0-c092-499c-9908-d6350e1cb99c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.240385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c58e26-4b9b-46ce-8e91-ee4df5e4982c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:56.281584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe07501-99de-4400-acd0-416fc3035138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:56.282272 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe07501-99de-4400-acd0-416fc3035138 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.282562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe07501-99de-4400-acd0-416fc3035138 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.282839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe07501-99de-4400-acd0-416fc3035138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.283692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe07501-99de-4400-acd0-416fc3035138 None admin] RBAC: Adding query filter params (role.id=2a5bf966af5a486fa21c10760ebfbcad, user.id=416443d4ca2d432aa30277d60a00c9ba, scope.system=all) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:56.284008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe07501-99de-4400-acd0-416fc3035138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:56.286782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe07501-99de-4400-acd0-416fc3035138 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:56.301599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 181/364] 10.222.0.23 () {56 vars in 1308 bytes} [Mon May 23 15:36:56 2022] GET /identity/v3/role_assignments?user.id=416443d4ca2d432aa30277d60a00c9ba&scope.system=all&role.id=2a5bf966af5a486fa21c10760ebfbcad => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:56.302943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30596318-9feb-4895-bfb9-16a28f463957 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.303233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30596318-9feb-4895-bfb9-16a28f463957 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.303561 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30596318-9feb-4895-bfb9-16a28f463957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.323207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-30596318-9feb-4895-bfb9-16a28f463957 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:56.324493 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-30596318-9feb-4895-bfb9-16a28f463957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:56.350684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fc8f0-c092-499c-9908-d6350e1cb99c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:56.437810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc7540-abf1-40a1-88e7-04c03031dfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:56.438444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 182/365] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:36:56.438812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc7540-abf1-40a1-88e7-04c03031dfab None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.439011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30596318-9feb-4895-bfb9-16a28f463957 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.439112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc7540-abf1-40a1-88e7-04c03031dfab None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.439349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc7540-abf1-40a1-88e7-04c03031dfab None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.441245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc7540-abf1-40a1-88e7-04c03031dfab None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:56.442723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc7540-abf1-40a1-88e7-04c03031dfab None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:56.448351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:56.448575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 184/366] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:56 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:56.449623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc7540-abf1-40a1-88e7-04c03031dfab None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.504208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:56.600164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30596318-9feb-4895-bfb9-16a28f463957 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:56.646132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc7540-abf1-40a1-88e7-04c03031dfab None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:56.656068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a2a8-ed9c-4f26-baf3-0b623c57a306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:56.658949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a2a8-ed9c-4f26-baf3-0b623c57a306 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.658949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a2a8-ed9c-4f26-baf3-0b623c57a306 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.658949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a2a8-ed9c-4f26-baf3-0b623c57a306 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.676987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512a2a8-ed9c-4f26-baf3-0b623c57a306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-0512a2a8-ed9c-4f26-baf3-0b623c57a306 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:56.685248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:56.686184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:56.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 183/367] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:36:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:56.687648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a2a8-ed9c-4f26-baf3-0b623c57a306 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.705513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ab6da-7b13-487f-9133-ddcb151d5bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:56.706905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ab6da-7b13-487f-9133-ddcb151d5bf3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.707604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ab6da-7b13-487f-9133-ddcb151d5bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.708099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ab6da-7b13-487f-9133-ddcb151d5bf3 None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.709657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ab6da-7b13-487f-9133-ddcb151d5bf3 None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:56.711459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ab6da-7b13-487f-9133-ddcb151d5bf3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:56.713761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 185/368] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:36:56 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:56.714928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ab6da-7b13-487f-9133-ddcb151d5bf3 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.764982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:56.800720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a2a8-ed9c-4f26-baf3-0b623c57a306 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:56.823192 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:56.827386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ab6da-7b13-487f-9133-ddcb151d5bf3 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:56.841528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1b02c-2ac8-490e-bcce-3f4cfe600819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:56.842988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1b02c-2ac8-490e-bcce-3f4cfe600819 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.843288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1b02c-2ac8-490e-bcce-3f4cfe600819 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.843570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1b02c-2ac8-490e-bcce-3f4cfe600819 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.854615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1b02c-2ac8-490e-bcce-3f4cfe600819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-18d1b02c-2ac8-490e-bcce-3f4cfe600819 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:56.862813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:56.863698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:56.864504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 184/369] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:56.865152 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bb7a0-8f29-4eaf-940b-5563fdf90fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:56.865322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1b02c-2ac8-490e-bcce-3f4cfe600819 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.866569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bb7a0-8f29-4eaf-940b-5563fdf90fad None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:56.866875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bb7a0-8f29-4eaf-940b-5563fdf90fad None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:56.867157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bb7a0-8f29-4eaf-940b-5563fdf90fad None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:56.867927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bb7a0-8f29-4eaf-940b-5563fdf90fad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:56.868239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bb7a0-8f29-4eaf-940b-5563fdf90fad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:56.869683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bb7a0-8f29-4eaf-940b-5563fdf90fad None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:56.878750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 186/370] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:36:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:56.879590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bb7a0-8f29-4eaf-940b-5563fdf90fad None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:56.992438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bb7a0-8f29-4eaf-940b-5563fdf90fad None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.026993 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1b02c-2ac8-490e-bcce-3f4cfe600819 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.030275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f9b45-ca60-4136-82ef-8182904dae32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.030956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f9b45-ca60-4136-82ef-8182904dae32 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.031243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f9b45-ca60-4136-82ef-8182904dae32 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.031522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f9b45-ca60-4136-82ef-8182904dae32 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.032275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f9b45-ca60-4136-82ef-8182904dae32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:57.032613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f9b45-ca60-4136-82ef-8182904dae32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.033980 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f9b45-ca60-4136-82ef-8182904dae32 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.043557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 187/371] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:57.048382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f9b45-ca60-4136-82ef-8182904dae32 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.061588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:57.071047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855933f0-d724-4ee0-b75c-3d6709e0898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.072069 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855933f0-d724-4ee0-b75c-3d6709e0898b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.072175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:57.072632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855933f0-d724-4ee0-b75c-3d6709e0898b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.078284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855933f0-d724-4ee0-b75c-3d6709e0898b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.078284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855933f0-d724-4ee0-b75c-3d6709e0898b None admin] RBAC: Adding query filter params (group.id=c61b7da72b3643489b6cb3990133a99a, role.id=177939861d6e4e82b48401be184d66b4, scope.project.id=682d8892f2d3403f862be249dc3a8acf) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:57.078284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855933f0-d724-4ee0-b75c-3d6709e0898b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.078284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855933f0-d724-4ee0-b75c-3d6709e0898b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.084136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 185/372] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 23 15:36:56 2022] GET /identity/v3/role_assignments?group.id=c61b7da72b3643489b6cb3990133a99a&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=177939861d6e4e82b48401be184d66b4 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:57.085220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855933f0-d724-4ee0-b75c-3d6709e0898b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.155692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f9b45-ca60-4136-82ef-8182904dae32 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.193390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172c7ea-1b80-496f-9349-e108ae3ad28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.194055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172c7ea-1b80-496f-9349-e108ae3ad28f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.194342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172c7ea-1b80-496f-9349-e108ae3ad28f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.194623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172c7ea-1b80-496f-9349-e108ae3ad28f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.282199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172c7ea-1b80-496f-9349-e108ae3ad28f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.283995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172c7ea-1b80-496f-9349-e108ae3ad28f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.312907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855933f0-d724-4ee0-b75c-3d6709e0898b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.313876 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:57.394645 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c89af-fe5b-440f-acab-9d973e02abb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.395860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c89af-fe5b-440f-acab-9d973e02abb3 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.396344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c89af-fe5b-440f-acab-9d973e02abb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.396835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c89af-fe5b-440f-acab-9d973e02abb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.402101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 188/373] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:57.407479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172c7ea-1b80-496f-9349-e108ae3ad28f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.502832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:57.516508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c89af-fe5b-440f-acab-9d973e02abb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.518319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c89af-fe5b-440f-acab-9d973e02abb3 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.530180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172c7ea-1b80-496f-9349-e108ae3ad28f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.565205 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:57.578225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f030d-3cd0-44b3-b8a5-f5e4557db55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.579194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f030d-3cd0-44b3-b8a5-f5e4557db55d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.579739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f030d-3cd0-44b3-b8a5-f5e4557db55d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.580020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f030d-3cd0-44b3-b8a5-f5e4557db55d None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.584516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f030d-3cd0-44b3-b8a5-f5e4557db55d None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.584516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f030d-3cd0-44b3-b8a5-f5e4557db55d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.586196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 189/374] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:57.591576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f030d-3cd0-44b3-b8a5-f5e4557db55d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.641235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 186/375] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:36:57.647040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c89af-fe5b-440f-acab-9d973e02abb3 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.706016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f030d-3cd0-44b3-b8a5-f5e4557db55d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.744620 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3970e5-5242-48e9-a942-14257fa35722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.745491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3970e5-5242-48e9-a942-14257fa35722 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.745803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3970e5-5242-48e9-a942-14257fa35722 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.746096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3970e5-5242-48e9-a942-14257fa35722 None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.747323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3970e5-5242-48e9-a942-14257fa35722 None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.749128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3970e5-5242-48e9-a942-14257fa35722 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.750899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 190/376] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:57.755237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:57.756646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3970e5-5242-48e9-a942-14257fa35722 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.768382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c89af-fe5b-440f-acab-9d973e02abb3 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.810743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f19cd-9419-4bb4-9eb0-3429ed542727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.811707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f19cd-9419-4bb4-9eb0-3429ed542727 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.812061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f19cd-9419-4bb4-9eb0-3429ed542727 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.812446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f19cd-9419-4bb4-9eb0-3429ed542727 None admin] PATH_INFO: `/v3/projects/6e65bbb7bace4fb6a7f71d68e850edbd` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.813832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f19cd-9419-4bb4-9eb0-3429ed542727 None admin] RBAC: Authorizing `identity:get_project(project_id=6e65bbb7bace4fb6a7f71d68e850edbd)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.815392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f19cd-9419-4bb4-9eb0-3429ed542727 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.818141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 187/377] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/projects/6e65bbb7bace4fb6a7f71d68e850edbd => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:57.823610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f19cd-9419-4bb4-9eb0-3429ed542727 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:57.831166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:57.888346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3970e5-5242-48e9-a942-14257fa35722 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.941658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee4e2c-54e2-4a68-819e-c6e06f775e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:57.942570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee4e2c-54e2-4a68-819e-c6e06f775e41 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:57.942998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee4e2c-54e2-4a68-819e-c6e06f775e41 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:57.943412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee4e2c-54e2-4a68-819e-c6e06f775e41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:57.944445 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee4e2c-54e2-4a68-819e-c6e06f775e41 None admin] RBAC: Adding query filter params (group.id=c61b7da72b3643489b6cb3990133a99a, role.id=f7e0a506c7894b20b2b65fedfd2244e5, scope.project.id=682d8892f2d3403f862be249dc3a8acf) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:57.945050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee4e2c-54e2-4a68-819e-c6e06f775e41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:57.946946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee4e2c-54e2-4a68-819e-c6e06f775e41 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:57.956995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f19cd-9419-4bb4-9eb0-3429ed542727 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:57.966153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 191/378] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/role_assignments?group.id=c61b7da72b3643489b6cb3990133a99a&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=f7e0a506c7894b20b2b65fedfd2244e5 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:58.004233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bc9e5-31b4-4a59-8c04-e0c243669bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:58.005169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc9e5-31b4-4a59-8c04-e0c243669bba None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.005454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc9e5-31b4-4a59-8c04-e0c243669bba None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.005732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc9e5-31b4-4a59-8c04-e0c243669bba None admin] PATH_INFO: `/v3/groups/1bb905d2b42341d6a21106d8c2d3b728` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.015206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bc9e5-31b4-4a59-8c04-e0c243669bba None admin] RBAC: Authorizing `identity:get_group(group_id=1bb905d2b42341d6a21106d8c2d3b728)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:58.016708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bc9e5-31b4-4a59-8c04-e0c243669bba None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:58.018437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 188/379] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:36:57 2022] GET /identity/v3/groups/1bb905d2b42341d6a21106d8c2d3b728 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:58.024732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee4e2c-54e2-4a68-819e-c6e06f775e41 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:58.148832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee4e2c-54e2-4a68-819e-c6e06f775e41 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:58.176447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:58.191995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b6e95-a062-422a-af35-a40af5b974e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:58.192705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b6e95-a062-422a-af35-a40af5b974e4 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.192990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b6e95-a062-422a-af35-a40af5b974e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.193268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b6e95-a062-422a-af35-a40af5b974e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.194124 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b6e95-a062-422a-af35-a40af5b974e4 None admin] RBAC: Adding query filter params (group.id=1bb905d2b42341d6a21106d8c2d3b728, role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=6e65bbb7bace4fb6a7f71d68e850edbd) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:58.194440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b6e95-a062-422a-af35-a40af5b974e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:58.195900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b6e95-a062-422a-af35-a40af5b974e4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:58.207189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 192/380] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 23 15:36:58 2022] GET /identity/v3/role_assignments?group.id=1bb905d2b42341d6a21106d8c2d3b728&scope.project.id=6e65bbb7bace4fb6a7f71d68e850edbd&role.id=04c11dea81274a69974ad5c639f2c60b => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:58.487977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0ea59-2ffc-4ff0-b8b6-df6f71bf4ae1 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.488506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0ea59-2ffc-4ff0-b8b6-df6f71bf4ae1 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.488506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0ea59-2ffc-4ff0-b8b6-df6f71bf4ae1 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.490151 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 189/381] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:36:58.496063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46e6c6-a8f8-48c2-8b6f-6fc9ba3d5906 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.496376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46e6c6-a8f8-48c2-8b6f-6fc9ba3d5906 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.496671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46e6c6-a8f8-48c2-8b6f-6fc9ba3d5906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.517920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-cc46e6c6-a8f8-48c2-8b6f-6fc9ba3d5906 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:58.519121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-cc46e6c6-a8f8-48c2-8b6f-6fc9ba3d5906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:58.595813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 193/382] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:36:58.636934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0ea59-2ffc-4ff0-b8b6-df6f71bf4ae1 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:58.676064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:58.789034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0ea59-2ffc-4ff0-b8b6-df6f71bf4ae1 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:58.833594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda8595-1b1a-489a-b5a1-daefdbade289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:58.834449 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda8595-1b1a-489a-b5a1-daefdbade289 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.834736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda8595-1b1a-489a-b5a1-daefdbade289 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.835008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda8595-1b1a-489a-b5a1-daefdbade289 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.855252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda8595-1b1a-489a-b5a1-daefdbade289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-0eda8595-1b1a-489a-b5a1-daefdbade289 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:58.863650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:58.864575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:36:58.865354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 190/383] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:36:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:36:58.874028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46e6c6-a8f8-48c2-8b6f-6fc9ba3d5906 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:58.911195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5d4a4-1da3-4667-83e7-2d1fbdf46e44 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.911530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5d4a4-1da3-4667-83e7-2d1fbdf46e44 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.911746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5d4a4-1da3-4667-83e7-2d1fbdf46e44 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.913607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 191/384] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:36:58.918616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40775306-4230-4ad8-a93e-3d3c15a2a608 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:58.918905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40775306-4230-4ad8-a93e-3d3c15a2a608 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:58.919185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40775306-4230-4ad8-a93e-3d3c15a2a608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:58.944412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-40775306-4230-4ad8-a93e-3d3c15a2a608 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:36:58.945660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-40775306-4230-4ad8-a93e-3d3c15a2a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:36:59.035296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 192/385] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 23 15:36:59.051068 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46e6c6-a8f8-48c2-8b6f-6fc9ba3d5906 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.076713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:59.095743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468f3eb-d029-407f-9ae1-d92840eef09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.096438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468f3eb-d029-407f-9ae1-d92840eef09b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.096725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468f3eb-d029-407f-9ae1-d92840eef09b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.096998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468f3eb-d029-407f-9ae1-d92840eef09b None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.097893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468f3eb-d029-407f-9ae1-d92840eef09b None admin] RBAC: Adding query filter params (name=member) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:59.098277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468f3eb-d029-407f-9ae1-d92840eef09b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.099772 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468f3eb-d029-407f-9ae1-d92840eef09b None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.109533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 194/386] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:36:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:59.115220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40775306-4230-4ad8-a93e-3d3c15a2a608 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.130575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:59.131957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468f3eb-d029-407f-9ae1-d92840eef09b None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.283648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40775306-4230-4ad8-a93e-3d3c15a2a608 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.325469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cddbb-efb9-433b-845b-6839d9d347cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.326258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cddbb-efb9-433b-845b-6839d9d347cb None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.326632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cddbb-efb9-433b-845b-6839d9d347cb None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.326920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cddbb-efb9-433b-845b-6839d9d347cb None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.328109 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cddbb-efb9-433b-845b-6839d9d347cb None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.329904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cddbb-efb9-433b-845b-6839d9d347cb None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.331830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 193/387] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:59.337076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cddbb-efb9-433b-845b-6839d9d347cb None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.394507 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468f3eb-d029-407f-9ae1-d92840eef09b None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.425318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:59.445059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cddbb-efb9-433b-845b-6839d9d347cb None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.449164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d9f69-965c-43cd-94b4-4f82b3f7018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.452126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d9f69-965c-43cd-94b4-4f82b3f7018c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.452126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d9f69-965c-43cd-94b4-4f82b3f7018c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.452126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d9f69-965c-43cd-94b4-4f82b3f7018c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.465542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d9f69-965c-43cd-94b4-4f82b3f7018c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-ea0d9f69-965c-43cd-94b4-4f82b3f7018c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:36:59.478949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:36:59.479831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:36:59.480658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 195/388] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:36:59.488189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2a9e0-2839-44e5-8451-7b2bd146b90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.489233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d9f69-965c-43cd-94b4-4f82b3f7018c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.490298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2a9e0-2839-44e5-8451-7b2bd146b90a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.490591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2a9e0-2839-44e5-8451-7b2bd146b90a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.490846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2a9e0-2839-44e5-8451-7b2bd146b90a None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.492386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2a9e0-2839-44e5-8451-7b2bd146b90a None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.493817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2a9e0-2839-44e5-8451-7b2bd146b90a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.495625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 194/389] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:36:59.513230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2a9e0-2839-44e5-8451-7b2bd146b90a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.573324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:59.691009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d9f69-965c-43cd-94b4-4f82b3f7018c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.704854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2a9e0-2839-44e5-8451-7b2bd146b90a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.761746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ddd3-7f36-40e2-8d80-721c8d54ad44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.762426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ddd3-7f36-40e2-8d80-721c8d54ad44 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.762708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ddd3-7f36-40e2-8d80-721c8d54ad44 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.762978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7ddd3-7f36-40e2-8d80-721c8d54ad44 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.763746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7ddd3-7f36-40e2-8d80-721c8d54ad44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:36:59.764608 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7ddd3-7f36-40e2-8d80-721c8d54ad44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.765970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7ddd3-7f36-40e2-8d80-721c8d54ad44 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.780116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f2f1f-4775-49f2-bace-b5069c30c73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.781044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f2f1f-4775-49f2-bace-b5069c30c73b None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.781321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f2f1f-4775-49f2-bace-b5069c30c73b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.781592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f2f1f-4775-49f2-bace-b5069c30c73b None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008/groups/c61b7da72b3643489b6cb3990133a99a/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.784452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f2f1f-4775-49f2-bace-b5069c30c73b None admin] RBAC: Authorizing `identity:create_grant(project_id=9811a47b7d5449f999b0f971ee26f008, group_id=c61b7da72b3643489b6cb3990133a99a, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.785897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f2f1f-4775-49f2-bace-b5069c30c73b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.788983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 196/390] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:59.798626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ddd3-7f36-40e2-8d80-721c8d54ad44 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:36:59.877934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 195/391] 10.222.0.23 () {58 vars in 1442 bytes} [Mon May 23 15:36:59 2022] PUT /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008/groups/c61b7da72b3643489b6cb3990133a99a/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:36:59.932421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7ddd3-7f36-40e2-8d80-721c8d54ad44 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:36:59.934453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:36:59.972524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8901080-84ae-4de6-bbeb-aa1c8c0a095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:36:59.973813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8901080-84ae-4de6-bbeb-aa1c8c0a095e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:36:59.974313 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8901080-84ae-4de6-bbeb-aa1c8c0a095e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:36:59.974815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8901080-84ae-4de6-bbeb-aa1c8c0a095e None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:36:59.976301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8901080-84ae-4de6-bbeb-aa1c8c0a095e None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:36:59.978045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8901080-84ae-4de6-bbeb-aa1c8c0a095e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:36:59.980198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 197/392] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:36:59.985499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f2f1f-4775-49f2-bace-b5069c30c73b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.035098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:00.100342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f2f1f-4775-49f2-bace-b5069c30c73b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.136935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb918cd-feca-4873-87f4-7c9c466d6b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:00.137863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb918cd-feca-4873-87f4-7c9c466d6b43 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.138156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb918cd-feca-4873-87f4-7c9c466d6b43 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.138364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb918cd-feca-4873-87f4-7c9c466d6b43 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.140664 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb918cd-feca-4873-87f4-7c9c466d6b43 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:00.142301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb918cd-feca-4873-87f4-7c9c466d6b43 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:00.144540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 196/393] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:36:59 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:00.153434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8901080-84ae-4de6-bbeb-aa1c8c0a095e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.234850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7553326-f638-4200-9f12-cc14fe131dfe None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.235147 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7553326-f638-4200-9f12-cc14fe131dfe None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.235387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7553326-f638-4200-9f12-cc14fe131dfe None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.237094 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 197/394] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:37:00.243181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcac453-62bb-42f8-b4b1-3526b5a3484c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.243739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcac453-62bb-42f8-b4b1-3526b5a3484c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.244217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcac453-62bb-42f8-b4b1-3526b5a3484c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.272451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-8dcac453-62bb-42f8-b4b1-3526b5a3484c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:00.274126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8dcac453-62bb-42f8-b4b1-3526b5a3484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:00.278332 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8901080-84ae-4de6-bbeb-aa1c8c0a095e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.320348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b978b1-e6d0-403b-9991-c839101870c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:00.321696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b978b1-e6d0-403b-9991-c839101870c2 None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.324756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b978b1-e6d0-403b-9991-c839101870c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.324756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b978b1-e6d0-403b-9991-c839101870c2 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008/groups/c61b7da72b3643489b6cb3990133a99a/roles/f7e0a506c7894b20b2b65fedfd2244e5` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.325742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b978b1-e6d0-403b-9991-c839101870c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9811a47b7d5449f999b0f971ee26f008, group_id=c61b7da72b3643489b6cb3990133a99a, role_id=f7e0a506c7894b20b2b65fedfd2244e5)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:00.327372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b978b1-e6d0-403b-9991-c839101870c2 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:00.380446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 198/395] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:00.401079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcac453-62bb-42f8-b4b1-3526b5a3484c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.402257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 198/396] 10.222.0.23 () {58 vars in 1442 bytes} [Mon May 23 15:37:00 2022] PUT /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008/groups/c61b7da72b3643489b6cb3990133a99a/roles/f7e0a506c7894b20b2b65fedfd2244e5 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:00.416288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:00.531762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcac453-62bb-42f8-b4b1-3526b5a3484c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.569756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b455-0dc5-478e-98e7-f94ded0ce0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:00.570561 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b455-0dc5-478e-98e7-f94ded0ce0de None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.570843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b455-0dc5-478e-98e7-f94ded0ce0de None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.571119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b455-0dc5-478e-98e7-f94ded0ce0de None admin] PATH_INFO: `/v3/roles/member` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.588991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715b455-0dc5-478e-98e7-f94ded0ce0de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-f715b455-0dc5-478e-98e7-f94ded0ce0de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:00.596944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:00.597872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:37:00.598720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:00.603278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 199/397] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:00.609543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b978b1-e6d0-403b-9991-c839101870c2 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.655538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:00.758652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b978b1-e6d0-403b-9991-c839101870c2 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.798254 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede6bd7-e83a-40f8-8254-470a0ce4b137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:00.798852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede6bd7-e83a-40f8-8254-470a0ce4b137 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.799136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede6bd7-e83a-40f8-8254-470a0ce4b137 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.799413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede6bd7-e83a-40f8-8254-470a0ce4b137 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.800168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede6bd7-e83a-40f8-8254-470a0ce4b137 None admin] RBAC: Adding query filter params (name=member) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:00.800499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede6bd7-e83a-40f8-8254-470a0ce4b137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:00.801856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede6bd7-e83a-40f8-8254-470a0ce4b137 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:00.810777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 199/398] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:37:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:00.816331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b455-0dc5-478e-98e7-f94ded0ce0de None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:00.895103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:00.941960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b455-0dc5-478e-98e7-f94ded0ce0de None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:00.962111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663e71d-4da7-4759-a1c4-2a7a0961f28a None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.962416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663e71d-4da7-4759-a1c4-2a7a0961f28a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.962671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663e71d-4da7-4759-a1c4-2a7a0961f28a None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.964341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 200/399] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:00.972414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc9567-e50b-4c72-b544-91a3454b0cd2 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:00.972716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc9567-e50b-4c72-b544-91a3454b0cd2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:00.972978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc9567-e50b-4c72-b544-91a3454b0cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:00.994418 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-13fc9567-e50b-4c72-b544-91a3454b0cd2 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:00.995652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-13fc9567-e50b-4c72-b544-91a3454b0cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:01.008935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207135bd-81c0-4a32-a9c5-5aff621f35b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.010075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207135bd-81c0-4a32-a9c5-5aff621f35b8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.010360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207135bd-81c0-4a32-a9c5-5aff621f35b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.010640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207135bd-81c0-4a32-a9c5-5aff621f35b8 None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.017136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207135bd-81c0-4a32-a9c5-5aff621f35b8 None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.018616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207135bd-81c0-4a32-a9c5-5aff621f35b8 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.020646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 200/400] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:37:00 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:01.026075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207135bd-81c0-4a32-a9c5-5aff621f35b8 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.072389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 201/401] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:37:01.097651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc9567-e50b-4c72-b544-91a3454b0cd2 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.124247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:01.213294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207135bd-81c0-4a32-a9c5-5aff621f35b8 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.217913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc9567-e50b-4c72-b544-91a3454b0cd2 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.248164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd8b82-0364-4339-b214-5fa8a33cb1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.249102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd8b82-0364-4339-b214-5fa8a33cb1e6 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.249388 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd8b82-0364-4339-b214-5fa8a33cb1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.249666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd8b82-0364-4339-b214-5fa8a33cb1e6 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.250855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd8b82-0364-4339-b214-5fa8a33cb1e6 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.252410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd8b82-0364-4339-b214-5fa8a33cb1e6 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.258033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53801fd-5f71-4a7d-a67d-92ccfa50b00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.258784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 201/402] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:01.258888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53801fd-5f71-4a7d-a67d-92ccfa50b00c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.259144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53801fd-5f71-4a7d-a67d-92ccfa50b00c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.259398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53801fd-5f71-4a7d-a67d-92ccfa50b00c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.260279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12bb51-4605-45b5-a36a-29bed3398e6c None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.260597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12bb51-4605-45b5-a36a-29bed3398e6c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.260877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12bb51-4605-45b5-a36a-29bed3398e6c None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.262653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 202/403] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:01.263509 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12bb51-4605-45b5-a36a-29bed3398e6c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.270005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:01.271657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53801fd-5f71-4a7d-a67d-92ccfa50b00c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-a53801fd-5f71-4a7d-a67d-92ccfa50b00c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:01.283272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:01.284190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:37:01.285037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:01.287220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 202/404] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:01.289122 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232c43c-3c83-4475-a962-ce4566e108d0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.289611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232c43c-3c83-4475-a962-ce4566e108d0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.290093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232c43c-3c83-4475-a962-ce4566e108d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.316216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2232c43c-3c83-4475-a962-ce4566e108d0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:01.317722 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2232c43c-3c83-4475-a962-ce4566e108d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:01.386937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12bb51-4605-45b5-a36a-29bed3398e6c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.393443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 203/405] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:01.394688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232c43c-3c83-4475-a962-ce4566e108d0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.413485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:01.446147 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5dfd4-f441-40e7-b216-c141e9bf8e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.447241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5dfd4-f441-40e7-b216-c141e9bf8e5a None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.447605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5dfd4-f441-40e7-b216-c141e9bf8e5a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.447980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5dfd4-f441-40e7-b216-c141e9bf8e5a None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf/groups/c61b7da72b3643489b6cb3990133a99a/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.450066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5dfd4-f441-40e7-b216-c141e9bf8e5a None admin] RBAC: Authorizing `identity:create_grant(project_id=682d8892f2d3403f862be249dc3a8acf, group_id=c61b7da72b3643489b6cb3990133a99a, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.452252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5dfd4-f441-40e7-b216-c141e9bf8e5a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.506311 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232c43c-3c83-4475-a962-ce4566e108d0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.524967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 203/406] 10.222.0.23 () {58 vars in 1442 bytes} [Mon May 23 15:37:01 2022] PUT /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf/groups/c61b7da72b3643489b6cb3990133a99a/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:01.526189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5dfd4-f441-40e7-b216-c141e9bf8e5a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.555155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7de0b-1b6e-4287-a314-a4083be247d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.556770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7de0b-1b6e-4287-a314-a4083be247d8 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.557228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7de0b-1b6e-4287-a314-a4083be247d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.557752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7de0b-1b6e-4287-a314-a4083be247d8 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.558676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7de0b-1b6e-4287-a314-a4083be247d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:01.559189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7de0b-1b6e-4287-a314-a4083be247d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.561378 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7de0b-1b6e-4287-a314-a4083be247d8 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.581801 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 204/407] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:01.589376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7de0b-1b6e-4287-a314-a4083be247d8 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.658355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:01.727350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:01.756681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7de0b-1b6e-4287-a314-a4083be247d8 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.761301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5dfd4-f441-40e7-b216-c141e9bf8e5a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.802267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21741e43-6beb-439b-b8da-21afd989333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.804009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741e43-6beb-439b-b8da-21afd989333b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.805230 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741e43-6beb-439b-b8da-21afd989333b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.805750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21741e43-6beb-439b-b8da-21afd989333b None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.807470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21741e43-6beb-439b-b8da-21afd989333b None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.807647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa5d1a-0ec1-4ddb-a3a8-49ef189f9422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.808416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa5d1a-0ec1-4ddb-a3a8-49ef189f9422 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.808705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa5d1a-0ec1-4ddb-a3a8-49ef189f9422 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.808989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa5d1a-0ec1-4ddb-a3a8-49ef189f9422 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.810144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21741e43-6beb-439b-b8da-21afd989333b None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.813280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 205/408] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:01.819622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21741e43-6beb-439b-b8da-21afd989333b None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.823863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa5d1a-0ec1-4ddb-a3a8-49ef189f9422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-1afa5d1a-0ec1-4ddb-a3a8-49ef189f9422 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:01.833303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:01.834190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:01.835000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 204/409] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:01.840890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa5d1a-0ec1-4ddb-a3a8-49ef189f9422 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:01.949287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21741e43-6beb-439b-b8da-21afd989333b None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.969921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa5d1a-0ec1-4ddb-a3a8-49ef189f9422 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:01.983762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2ba7f-8539-4cf0-9eb6-f5578d242ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:01.984658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2ba7f-8539-4cf0-9eb6-f5578d242ee0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:01.984938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2ba7f-8539-4cf0-9eb6-f5578d242ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:01.985209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2ba7f-8539-4cf0-9eb6-f5578d242ee0 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:01.986393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2ba7f-8539-4cf0-9eb6-f5578d242ee0 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:01.987826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2ba7f-8539-4cf0-9eb6-f5578d242ee0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:01.989881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 206/410] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:02.001654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2ba7f-8539-4cf0-9eb6-f5578d242ee0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:02.004532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:02.018005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6a0e5-6a74-4be5-9ddf-fcdfaeea2c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:02.019050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6a0e5-6a74-4be5-9ddf-fcdfaeea2c44 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:02.019591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6a0e5-6a74-4be5-9ddf-fcdfaeea2c44 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:02.020094 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6a0e5-6a74-4be5-9ddf-fcdfaeea2c44 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:02.020933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:02.022037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6a0e5-6a74-4be5-9ddf-fcdfaeea2c44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:02.022557 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6a0e5-6a74-4be5-9ddf-fcdfaeea2c44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:02.024226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6a0e5-6a74-4be5-9ddf-fcdfaeea2c44 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:02.034869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 205/411] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:02.041785 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6a0e5-6a74-4be5-9ddf-fcdfaeea2c44 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:02.184544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6a0e5-6a74-4be5-9ddf-fcdfaeea2c44 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:02.207205 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2ba7f-8539-4cf0-9eb6-f5578d242ee0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:02.222125 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43b378-a190-4fb4-907e-b8492665f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:02.223325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43b378-a190-4fb4-907e-b8492665f023 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:02.223817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43b378-a190-4fb4-907e-b8492665f023 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:02.224318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43b378-a190-4fb4-907e-b8492665f023 None admin] PATH_INFO: `/v3/groups/1bb905d2b42341d6a21106d8c2d3b728` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:02.225764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43b378-a190-4fb4-907e-b8492665f023 None admin] RBAC: Authorizing `identity:get_group(group_id=1bb905d2b42341d6a21106d8c2d3b728)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:02.227433 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43b378-a190-4fb4-907e-b8492665f023 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:02.229325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 206/412] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:37:02 2022] GET /identity/v3/groups/1bb905d2b42341d6a21106d8c2d3b728 => generated 229 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:02.236623 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43b378-a190-4fb4-907e-b8492665f023 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:02.268192 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf97d0-3762-4014-9496-5afd002ef48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:02.269296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf97d0-3762-4014-9496-5afd002ef48d None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:02.269583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf97d0-3762-4014-9496-5afd002ef48d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:02.269936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf97d0-3762-4014-9496-5afd002ef48d None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf/groups/c61b7da72b3643489b6cb3990133a99a/roles/f7e0a506c7894b20b2b65fedfd2244e5` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:02.275898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf97d0-3762-4014-9496-5afd002ef48d None admin] RBAC: Authorizing `identity:create_grant(project_id=682d8892f2d3403f862be249dc3a8acf, group_id=c61b7da72b3643489b6cb3990133a99a, role_id=f7e0a506c7894b20b2b65fedfd2244e5)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:02.277456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacf97d0-3762-4014-9496-5afd002ef48d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:02.315031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:02.362539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 207/413] 10.222.0.23 () {58 vars in 1442 bytes} [Mon May 23 15:37:02 2022] PUT /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf/groups/c61b7da72b3643489b6cb3990133a99a/roles/f7e0a506c7894b20b2b65fedfd2244e5 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:37:02.372167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43b378-a190-4fb4-907e-b8492665f023 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:02.423579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331e385-e814-4340-882f-a0a4d926f9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:02.424826 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331e385-e814-4340-882f-a0a4d926f9fe None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:02.425321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331e385-e814-4340-882f-a0a4d926f9fe None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:02.425749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331e385-e814-4340-882f-a0a4d926f9fe None admin] PATH_INFO: `/v3/projects/6e65bbb7bace4fb6a7f71d68e850edbd` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:02.427208 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331e385-e814-4340-882f-a0a4d926f9fe None admin] RBAC: Authorizing `identity:get_project(project_id=6e65bbb7bace4fb6a7f71d68e850edbd)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:02.428995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331e385-e814-4340-882f-a0a4d926f9fe None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:02.431344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 207/414] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:37:02 2022] GET /identity/v3/projects/6e65bbb7bace4fb6a7f71d68e850edbd => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:02.436693 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf97d0-3762-4014-9496-5afd002ef48d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:02.537839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:02.578637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf97d0-3762-4014-9496-5afd002ef48d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:02.617492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100a2ba-dbf4-428b-ae27-6599184408ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:02.618499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100a2ba-dbf4-428b-ae27-6599184408ed None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:02.618856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100a2ba-dbf4-428b-ae27-6599184408ed None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:02.619209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100a2ba-dbf4-428b-ae27-6599184408ed None admin] PATH_INFO: `/v3/projects/6e65bbb7bace4fb6a7f71d68e850edbd/groups/1bb905d2b42341d6a21106d8c2d3b728/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:02.621299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100a2ba-dbf4-428b-ae27-6599184408ed None admin] RBAC: Authorizing `identity:create_grant(project_id=6e65bbb7bace4fb6a7f71d68e850edbd, group_id=1bb905d2b42341d6a21106d8c2d3b728, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:02.622756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100a2ba-dbf4-428b-ae27-6599184408ed None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:02.694803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 208/415] 10.222.0.23 () {58 vars in 1442 bytes} [Mon May 23 15:37:02 2022] PUT /identity/v3/projects/6e65bbb7bace4fb6a7f71d68e850edbd/groups/1bb905d2b42341d6a21106d8c2d3b728/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:02.758586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002d3ad-0bef-4be6-b98e-4b4611f6694f None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:02.759097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002d3ad-0bef-4be6-b98e-4b4611f6694f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:02.759168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002d3ad-0bef-4be6-b98e-4b4611f6694f None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:02.761243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 208/416] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:02.767182 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72de9e-2c30-460b-9d56-2aad52905d18 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:02.767475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72de9e-2c30-460b-9d56-2aad52905d18 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:02.767755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72de9e-2c30-460b-9d56-2aad52905d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:02.793499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-dd72de9e-2c30-460b-9d56-2aad52905d18 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:02.795071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dd72de9e-2c30-460b-9d56-2aad52905d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:02.880308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 209/417] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:37:02.918585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002d3ad-0bef-4be6-b98e-4b4611f6694f None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:03.036573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002d3ad-0bef-4be6-b98e-4b4611f6694f None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:03.058776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:03.073442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834cc66-a212-4e50-9247-40f156e7ce14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:03.074252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834cc66-a212-4e50-9247-40f156e7ce14 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:03.074541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834cc66-a212-4e50-9247-40f156e7ce14 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:03.074817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834cc66-a212-4e50-9247-40f156e7ce14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:03.085953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834cc66-a212-4e50-9247-40f156e7ce14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-6834cc66-a212-4e50-9247-40f156e7ce14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:03.093753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:03.094632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:03.095411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 209/418] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:03.100852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72de9e-2c30-460b-9d56-2aad52905d18 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:03.209000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:03.221023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847809f8-2a4c-4358-ace8-7dfa256a3ae0 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:03.221322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847809f8-2a4c-4358-ace8-7dfa256a3ae0 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:03.221603 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847809f8-2a4c-4358-ace8-7dfa256a3ae0 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:03.224269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 210/419] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:37:03.232400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523c0a9-1e52-4c7a-bce9-f2995e084a67 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:03.232400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523c0a9-1e52-4c7a-bce9-f2995e084a67 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:03.232400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523c0a9-1e52-4c7a-bce9-f2995e084a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:03.254828 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-3523c0a9-1e52-4c7a-bce9-f2995e084a67 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:03.256083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3523c0a9-1e52-4c7a-bce9-f2995e084a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:03.277627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72de9e-2c30-460b-9d56-2aad52905d18 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:03.338496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fdeb0-2747-4d6d-ad29-2d762022e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:03.339496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fdeb0-2747-4d6d-ad29-2d762022e160 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:03.339954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fdeb0-2747-4d6d-ad29-2d762022e160 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:03.340573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fdeb0-2747-4d6d-ad29-2d762022e160 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:03.341517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810fdeb0-2747-4d6d-ad29-2d762022e160 None admin] RBAC: Adding query filter params (name=member) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:03.343688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810fdeb0-2747-4d6d-ad29-2d762022e160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:03.345288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810fdeb0-2747-4d6d-ad29-2d762022e160 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:03.359991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 210/420] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:37:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:03.366200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fdeb0-2747-4d6d-ad29-2d762022e160 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:03.400463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 211/421] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:03.427761 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523c0a9-1e52-4c7a-bce9-f2995e084a67 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:03.509451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fdeb0-2747-4d6d-ad29-2d762022e160 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:03.536760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:03.553188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2d1b3-89a5-4a6a-8bf7-a0b300b250ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:03.554170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2d1b3-89a5-4a6a-8bf7-a0b300b250ec None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:03.554614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2d1b3-89a5-4a6a-8bf7-a0b300b250ec None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:03.559894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2d1b3-89a5-4a6a-8bf7-a0b300b250ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:03.578051 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523c0a9-1e52-4c7a-bce9-f2995e084a67 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:03.593340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:03.637563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd5161-e1c0-4809-89b6-4ddc27345b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:03.638380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd5161-e1c0-4809-89b6-4ddc27345b6a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:03.638665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd5161-e1c0-4809-89b6-4ddc27345b6a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:03.638973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd5161-e1c0-4809-89b6-4ddc27345b6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:03.660070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd5161-e1c0-4809-89b6-4ddc27345b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-66cd5161-e1c0-4809-89b6-4ddc27345b6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:03.668286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:03.669174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:37:03.669961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:03.671169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 212/422] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:03.679258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd5161-e1c0-4809-89b6-4ddc27345b6a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:03.693916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2d1b3-89a5-4a6a-8bf7-a0b300b250ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:03.695394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2d1b3-89a5-4a6a-8bf7-a0b300b250ec None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:03.866682 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 211/423] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:03.872747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd5161-e1c0-4809-89b6-4ddc27345b6a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:03.874267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2d1b3-89a5-4a6a-8bf7-a0b300b250ec None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:03.915020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:03.943380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296b9e4-6196-48b3-b894-4163720e084c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:03.944421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296b9e4-6196-48b3-b894-4163720e084c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:03.944899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296b9e4-6196-48b3-b894-4163720e084c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:03.945389 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296b9e4-6196-48b3-b894-4163720e084c None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:03.946337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296b9e4-6196-48b3-b894-4163720e084c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:03.946836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296b9e4-6196-48b3-b894-4163720e084c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:03.949662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296b9e4-6196-48b3-b894-4163720e084c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:03.965337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 213/424] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:03.970989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296b9e4-6196-48b3-b894-4163720e084c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:04.025918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:04.060626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2d1b3-89a5-4a6a-8bf7-a0b300b250ec None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:04.135150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973affb-b194-4126-bf4d-0bc28c315ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:04.136512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973affb-b194-4126-bf4d-0bc28c315ca3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:04.137767 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973affb-b194-4126-bf4d-0bc28c315ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:04.138863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973affb-b194-4126-bf4d-0bc28c315ca3 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:04.142047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973affb-b194-4126-bf4d-0bc28c315ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:04.144574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973affb-b194-4126-bf4d-0bc28c315ca3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:04.149372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296b9e4-6196-48b3-b894-4163720e084c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:04.151037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 212/425] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:37:03 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:04.152453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973affb-b194-4126-bf4d-0bc28c315ca3 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:04.202758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78826e51-e1d3-4bdc-a389-cfd404e34e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:04.203777 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78826e51-e1d3-4bdc-a389-cfd404e34e81 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:04.204262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78826e51-e1d3-4bdc-a389-cfd404e34e81 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:04.204796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78826e51-e1d3-4bdc-a389-cfd404e34e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:04.302971 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78826e51-e1d3-4bdc-a389-cfd404e34e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:04.304452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78826e51-e1d3-4bdc-a389-cfd404e34e81 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:04.346024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:04.363549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:04.374680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973affb-b194-4126-bf4d-0bc28c315ca3 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:04.443046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 214/426] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 15:37:04.448280 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78826e51-e1d3-4bdc-a389-cfd404e34e81 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:04.474418 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3abfbb-b09c-48b9-b69c-e07b06896250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:04.475293 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3abfbb-b09c-48b9-b69c-e07b06896250 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:04.475583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3abfbb-b09c-48b9-b69c-e07b06896250 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:04.475863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3abfbb-b09c-48b9-b69c-e07b06896250 None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:04.479452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3abfbb-b09c-48b9-b69c-e07b06896250 None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:04.480951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3abfbb-b09c-48b9-b69c-e07b06896250 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:04.483445 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 213/427] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:37:04 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:04.493600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3abfbb-b09c-48b9-b69c-e07b06896250 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:04.554850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78826e51-e1d3-4bdc-a389-cfd404e34e81 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:04.586137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:04.592042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d947a-49e4-463f-bab6-bf0c94ab4709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:04.593796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d947a-49e4-463f-bab6-bf0c94ab4709 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:04.594552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d947a-49e4-463f-bab6-bf0c94ab4709 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:04.599775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d947a-49e4-463f-bab6-bf0c94ab4709 None admin] PATH_INFO: `/v3/projects/9811a47b7d5449f999b0f971ee26f008` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:04.599775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d947a-49e4-463f-bab6-bf0c94ab4709 None admin] RBAC: Authorizing `identity:get_project(project_id=9811a47b7d5449f999b0f971ee26f008)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:04.599775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d947a-49e4-463f-bab6-bf0c94ab4709 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:04.601324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 215/428] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:37:04 2022] GET /identity/v3/projects/9811a47b7d5449f999b0f971ee26f008 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:04.602512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11585-6259-4eca-8ce0-5f5f3efee9cd None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:04.602854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11585-6259-4eca-8ce0-5f5f3efee9cd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:04.603135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb11585-6259-4eca-8ce0-5f5f3efee9cd None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:04.604344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 216/429] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:04.607055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb11585-6259-4eca-8ce0-5f5f3efee9cd None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:04.641824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3abfbb-b09c-48b9-b69c-e07b06896250 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:04.679567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0ca55-1c02-44cb-9ab7-5cd7d89d3bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:04.680266 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0ca55-1c02-44cb-9ab7-5cd7d89d3bee None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:04.680653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0ca55-1c02-44cb-9ab7-5cd7d89d3bee None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:04.680929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0ca55-1c02-44cb-9ab7-5cd7d89d3bee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:04.681977 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0ca55-1c02-44cb-9ab7-5cd7d89d3bee None admin] RBAC: Adding query filter params (group.id=c61b7da72b3643489b6cb3990133a99a, role.id=177939861d6e4e82b48401be184d66b4, scope.project.id=9811a47b7d5449f999b0f971ee26f008) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:04.682882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0ca55-1c02-44cb-9ab7-5cd7d89d3bee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:04.684719 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0ca55-1c02-44cb-9ab7-5cd7d89d3bee None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:04.693266 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 214/430] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 23 15:37:04 2022] GET /identity/v3/role_assignments?group.id=c61b7da72b3643489b6cb3990133a99a&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=177939861d6e4e82b48401be184d66b4 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:04.697185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed206e-059b-415c-82e6-595af9c2bc74 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:04.697495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed206e-059b-415c-82e6-595af9c2bc74 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:04.700402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed206e-059b-415c-82e6-595af9c2bc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:04.734284 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-5aed206e-059b-415c-82e6-595af9c2bc74 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:04.736149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5aed206e-059b-415c-82e6-595af9c2bc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:04.738200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb11585-6259-4eca-8ce0-5f5f3efee9cd None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:04.770267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:04.804424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dafa0-c594-496d-974c-0a1d67f9d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:04.805320 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dafa0-c594-496d-974c-0a1d67f9d67d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:04.806247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dafa0-c594-496d-974c-0a1d67f9d67d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:04.807951 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dafa0-c594-496d-974c-0a1d67f9d67d None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:04.810158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dafa0-c594-496d-974c-0a1d67f9d67d None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:04.816856 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dafa0-c594-496d-974c-0a1d67f9d67d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:04.818739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 217/431] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:37:04 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:04.827666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dafa0-c594-496d-974c-0a1d67f9d67d None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:04.876737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 215/432] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:04.891376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e31b6-2a4e-42a7-ba73-3be101c7a676 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:04.891729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e31b6-2a4e-42a7-ba73-3be101c7a676 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:04.891975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e31b6-2a4e-42a7-ba73-3be101c7a676 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:04.894118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 216/433] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:37:04.900288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec428e6-044b-4908-b7f6-2a58a9e7bdaa None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:04.900601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec428e6-044b-4908-b7f6-2a58a9e7bdaa None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:04.900977 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec428e6-044b-4908-b7f6-2a58a9e7bdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:04.923593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-3ec428e6-044b-4908-b7f6-2a58a9e7bdaa None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:04.924857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3ec428e6-044b-4908-b7f6-2a58a9e7bdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:04.939245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:04.978640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dafa0-c594-496d-974c-0a1d67f9d67d None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:05.017902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 217/434] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:37:05.018568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec428e6-044b-4908-b7f6-2a58a9e7bdaa None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:05.022598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46255e-be56-4161-942a-cf9aa8ed704f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:05.023269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46255e-be56-4161-942a-cf9aa8ed704f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:05.023550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46255e-be56-4161-942a-cf9aa8ed704f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:05.023826 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46255e-be56-4161-942a-cf9aa8ed704f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:05.024785 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46255e-be56-4161-942a-cf9aa8ed704f None admin] RBAC: Adding query filter params (group.id=c61b7da72b3643489b6cb3990133a99a, role.id=f7e0a506c7894b20b2b65fedfd2244e5, scope.project.id=9811a47b7d5449f999b0f971ee26f008) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:05.025152 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46255e-be56-4161-942a-cf9aa8ed704f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:05.026608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46255e-be56-4161-942a-cf9aa8ed704f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:05.036003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 218/435] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 23 15:37:04 2022] GET /identity/v3/role_assignments?group.id=c61b7da72b3643489b6cb3990133a99a&scope.project.id=9811a47b7d5449f999b0f971ee26f008&role.id=f7e0a506c7894b20b2b65fedfd2244e5 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:05.048093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46255e-be56-4161-942a-cf9aa8ed704f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:05.125390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:05.148068 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec428e6-044b-4908-b7f6-2a58a9e7bdaa None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:05.172159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46255e-be56-4161-942a-cf9aa8ed704f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:05.192866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0539ee3-64d1-49c4-b577-d9f2493cb50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:05.193698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0539ee3-64d1-49c4-b577-d9f2493cb50e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:05.193983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0539ee3-64d1-49c4-b577-d9f2493cb50e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:05.194263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0539ee3-64d1-49c4-b577-d9f2493cb50e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:05.214600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0539ee3-64d1-49c4-b577-d9f2493cb50e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:05.219040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-e0539ee3-64d1-49c4-b577-d9f2493cb50e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:05.222581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:05.223494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:05.224289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 218/436] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:05.229995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba20572-d32b-490f-9ed3-3db457144ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:05.230468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba20572-d32b-490f-9ed3-3db457144ff6 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:05.230754 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba20572-d32b-490f-9ed3-3db457144ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:05.231033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba20572-d32b-490f-9ed3-3db457144ff6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:05.232555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0539ee3-64d1-49c4-b577-d9f2493cb50e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:05.251200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba20572-d32b-490f-9ed3-3db457144ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-2ba20572-d32b-490f-9ed3-3db457144ff6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:05.262591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:05.263791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:37:05.265340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:05.268864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 219/437] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:05.275189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba20572-d32b-490f-9ed3-3db457144ff6 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:05.384295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0539ee3-64d1-49c4-b577-d9f2493cb50e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:05.414005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba20572-d32b-490f-9ed3-3db457144ff6 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:05.429163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27437c-1c84-4010-b20b-a7c210e5e450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:05.429857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27437c-1c84-4010-b20b-a7c210e5e450 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:05.430138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27437c-1c84-4010-b20b-a7c210e5e450 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:05.430412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27437c-1c84-4010-b20b-a7c210e5e450 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:05.431309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27437c-1c84-4010-b20b-a7c210e5e450 None admin] RBAC: Adding query filter params (name=member) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:05.432085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27437c-1c84-4010-b20b-a7c210e5e450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:05.433538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27437c-1c84-4010-b20b-a7c210e5e450 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:05.443656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 219/438] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:37:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:05.444860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633dac6-88bb-47a1-b910-7f1e2351d257 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:05.445137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633dac6-88bb-47a1-b910-7f1e2351d257 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:05.445417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633dac6-88bb-47a1-b910-7f1e2351d257 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:05.446783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 220/439] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:05.451419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633dac6-88bb-47a1-b910-7f1e2351d257 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:05.455666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:05.458369 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56827711-d423-4256-9f84-e462dd93b0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:05.459061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56827711-d423-4256-9f84-e462dd93b0c2 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:05.459348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56827711-d423-4256-9f84-e462dd93b0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:05.459627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56827711-d423-4256-9f84-e462dd93b0c2 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:05.460416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56827711-d423-4256-9f84-e462dd93b0c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:05.460730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56827711-d423-4256-9f84-e462dd93b0c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:05.462153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56827711-d423-4256-9f84-e462dd93b0c2 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:05.470920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 220/440] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:05.472348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d07ab0-55ac-44b2-9727-15768d28fe51 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:05.472876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d07ab0-55ac-44b2-9727-15768d28fe51 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:05.473294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d07ab0-55ac-44b2-9727-15768d28fe51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:05.490467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-64d07ab0-55ac-44b2-9727-15768d28fe51 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:05.491611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-64d07ab0-55ac-44b2-9727-15768d28fe51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:05.554252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:05.572633 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633dac6-88bb-47a1-b910-7f1e2351d257 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:05.574965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 221/441] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:05.576041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d07ab0-55ac-44b2-9727-15768d28fe51 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:05.610862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e160f1f-fe05-4b76-a2e9-d39adc1a53f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:05.611533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e160f1f-fe05-4b76-a2e9-d39adc1a53f6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:05.611810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e160f1f-fe05-4b76-a2e9-d39adc1a53f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:05.612086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e160f1f-fe05-4b76-a2e9-d39adc1a53f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:05.679243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e160f1f-fe05-4b76-a2e9-d39adc1a53f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:05.680705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d07ab0-55ac-44b2-9727-15768d28fe51 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:05.680849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e160f1f-fe05-4b76-a2e9-d39adc1a53f6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:05.683290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:05.715555 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18982a81-f348-4c1d-939f-d42f1eab8066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:05.716207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18982a81-f348-4c1d-939f-d42f1eab8066 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:05.716530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18982a81-f348-4c1d-939f-d42f1eab8066 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:05.716811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18982a81-f348-4c1d-939f-d42f1eab8066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:05.769328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:05.777741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 221/442] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:05.778569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e160f1f-fe05-4b76-a2e9-d39adc1a53f6 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:05.785571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18982a81-f348-4c1d-939f-d42f1eab8066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:05.787038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18982a81-f348-4c1d-939f-d42f1eab8066 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:05.874041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e160f1f-fe05-4b76-a2e9-d39adc1a53f6 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:05.876238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:05.883717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 222/443] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:05.884562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18982a81-f348-4c1d-939f-d42f1eab8066 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:05.908619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf3f7b-7bdf-4d57-92bb-c3c28924cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:05.909416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf3f7b-7bdf-4d57-92bb-c3c28924cd62 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:05.909709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf3f7b-7bdf-4d57-92bb-c3c28924cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:05.909986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf3f7b-7bdf-4d57-92bb-c3c28924cd62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:05.920111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf3f7b-7bdf-4d57-92bb-c3c28924cd62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-1ccf3f7b-7bdf-4d57-92bb-c3c28924cd62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:05.927653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:05.929115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 222/444] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:05.929911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf3f7b-7bdf-4d57-92bb-c3c28924cd62 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:05.969993 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:05.985393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18982a81-f348-4c1d-939f-d42f1eab8066 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:06.019722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766d5d5-b211-466c-b8fc-685ad1ef24d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:06.020552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766d5d5-b211-466c-b8fc-685ad1ef24d1 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:06.020841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766d5d5-b211-466c-b8fc-685ad1ef24d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:06.021127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766d5d5-b211-466c-b8fc-685ad1ef24d1 None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:06.022317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766d5d5-b211-466c-b8fc-685ad1ef24d1 None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:06.023775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766d5d5-b211-466c-b8fc-685ad1ef24d1 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:06.025649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 223/445] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:37:05 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:06.026248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766d5d5-b211-466c-b8fc-685ad1ef24d1 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:06.030971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf3f7b-7bdf-4d57-92bb-c3c28924cd62 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:06.064297 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade595a-d3e9-4b28-977a-3796bb7fc0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:06.065174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade595a-d3e9-4b28-977a-3796bb7fc0ca None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:06.065456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade595a-d3e9-4b28-977a-3796bb7fc0ca None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:06.065728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade595a-d3e9-4b28-977a-3796bb7fc0ca None admin] PATH_INFO: `/v3/projects/682d8892f2d3403f862be249dc3a8acf` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:06.066903 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade595a-d3e9-4b28-977a-3796bb7fc0ca None admin] RBAC: Authorizing `identity:get_project(project_id=682d8892f2d3403f862be249dc3a8acf)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:06.068309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade595a-d3e9-4b28-977a-3796bb7fc0ca None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:06.069985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 223/446] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:37:05 2022] GET /identity/v3/projects/682d8892f2d3403f862be249dc3a8acf => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:06.070727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade595a-d3e9-4b28-977a-3796bb7fc0ca None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:06.102072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:06.126545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766d5d5-b211-466c-b8fc-685ad1ef24d1 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:06.160904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf535e-44ca-478c-a8dd-01c03db0af7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:06.161572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf535e-44ca-478c-a8dd-01c03db0af7a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:06.161860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf535e-44ca-478c-a8dd-01c03db0af7a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:06.162141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf535e-44ca-478c-a8dd-01c03db0af7a None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:06.162889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf535e-44ca-478c-a8dd-01c03db0af7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:06.163202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf535e-44ca-478c-a8dd-01c03db0af7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:06.164609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf535e-44ca-478c-a8dd-01c03db0af7a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:06.168843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade595a-d3e9-4b28-977a-3796bb7fc0ca None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:06.172577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 224/447] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:06.173372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf535e-44ca-478c-a8dd-01c03db0af7a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:06.192849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:06.205390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc561cb-8fba-479e-9c04-7273a0a3376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:06.206242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc561cb-8fba-479e-9c04-7273a0a3376d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:06.206524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc561cb-8fba-479e-9c04-7273a0a3376d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:06.206804 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc561cb-8fba-479e-9c04-7273a0a3376d None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:06.207981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc561cb-8fba-479e-9c04-7273a0a3376d None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:06.209494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc561cb-8fba-479e-9c04-7273a0a3376d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:06.211107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 224/448] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:37:06 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:06.211855 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc561cb-8fba-479e-9c04-7273a0a3376d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:06.279165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf535e-44ca-478c-a8dd-01c03db0af7a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:06.313320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:06.314442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c10626-fc65-4922-8d12-d41627155342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:06.315313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c10626-fc65-4922-8d12-d41627155342 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:06.315598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c10626-fc65-4922-8d12-d41627155342 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:06.315874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c10626-fc65-4922-8d12-d41627155342 None admin] PATH_INFO: `/v3/groups/c61b7da72b3643489b6cb3990133a99a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:06.316403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc561cb-8fba-479e-9c04-7273a0a3376d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:06.317064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c10626-fc65-4922-8d12-d41627155342 None admin] RBAC: Authorizing `identity:get_group(group_id=c61b7da72b3643489b6cb3990133a99a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:06.318495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c10626-fc65-4922-8d12-d41627155342 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:06.320096 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 225/449] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:37:06 2022] GET /identity/v3/groups/c61b7da72b3643489b6cb3990133a99a => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:06.320892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c10626-fc65-4922-8d12-d41627155342 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:06.349668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b1f0a-b98e-44d4-990a-51121bbbffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:06.350316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b1f0a-b98e-44d4-990a-51121bbbffa4 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:06.350600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b1f0a-b98e-44d4-990a-51121bbbffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:06.350880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b1f0a-b98e-44d4-990a-51121bbbffa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:06.405264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:06.417665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b1f0a-b98e-44d4-990a-51121bbbffa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:06.419334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b1f0a-b98e-44d4-990a-51121bbbffa4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:06.422897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c10626-fc65-4922-8d12-d41627155342 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:06.456670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aba99-a1c0-4ccb-8f94-9f055ca1d846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:06.457324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aba99-a1c0-4ccb-8f94-9f055ca1d846 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:06.457610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aba99-a1c0-4ccb-8f94-9f055ca1d846 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:06.457893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aba99-a1c0-4ccb-8f94-9f055ca1d846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:06.458723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aba99-a1c0-4ccb-8f94-9f055ca1d846 None admin] RBAC: Adding query filter params (group.id=c61b7da72b3643489b6cb3990133a99a, role.id=177939861d6e4e82b48401be184d66b4, scope.project.id=682d8892f2d3403f862be249dc3a8acf) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:06.459037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aba99-a1c0-4ccb-8f94-9f055ca1d846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:06.460494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aba99-a1c0-4ccb-8f94-9f055ca1d846 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:06.466291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 226/450] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 23 15:37:06 2022] GET /identity/v3/role_assignments?group.id=c61b7da72b3643489b6cb3990133a99a&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=177939861d6e4e82b48401be184d66b4 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:06.467171 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aba99-a1c0-4ccb-8f94-9f055ca1d846 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:06.533913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:06.542149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 225/451] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:06.543555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b1f0a-b98e-44d4-990a-51121bbbffa4 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:06.588658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aba99-a1c0-4ccb-8f94-9f055ca1d846 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:06.624975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc5cd5-1d0e-491e-8140-9b1bd47dc813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:06.625711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc5cd5-1d0e-491e-8140-9b1bd47dc813 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:06.625998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc5cd5-1d0e-491e-8140-9b1bd47dc813 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:06.626276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc5cd5-1d0e-491e-8140-9b1bd47dc813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:06.627119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc5cd5-1d0e-491e-8140-9b1bd47dc813 None admin] RBAC: Adding query filter params (group.id=c61b7da72b3643489b6cb3990133a99a, role.id=f7e0a506c7894b20b2b65fedfd2244e5, scope.project.id=682d8892f2d3403f862be249dc3a8acf) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:06.627432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc5cd5-1d0e-491e-8140-9b1bd47dc813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:06.628946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc5cd5-1d0e-491e-8140-9b1bd47dc813 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:06.636415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 227/452] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 23 15:37:06 2022] GET /identity/v3/role_assignments?group.id=c61b7da72b3643489b6cb3990133a99a&scope.project.id=682d8892f2d3403f862be249dc3a8acf&role.id=f7e0a506c7894b20b2b65fedfd2244e5 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:06.661178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b1f0a-b98e-44d4-990a-51121bbbffa4 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:06.700600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a86e9-b398-4db5-a3e3-57221c96d3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:06.701492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a86e9-b398-4db5-a3e3-57221c96d3b2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:06.701812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a86e9-b398-4db5-a3e3-57221c96d3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:06.702089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a86e9-b398-4db5-a3e3-57221c96d3b2 None admin] PATH_INFO: `/v3/projects/6e65bbb7bace4fb6a7f71d68e850edbd` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:06.703425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a86e9-b398-4db5-a3e3-57221c96d3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=6e65bbb7bace4fb6a7f71d68e850edbd)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:06.704962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a86e9-b398-4db5-a3e3-57221c96d3b2 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:06.706776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 226/453] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:37:06 2022] GET /identity/v3/projects/6e65bbb7bace4fb6a7f71d68e850edbd => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:06.712576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc5cd5-1d0e-491e-8140-9b1bd47dc813 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:06.727432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:06.827533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc5cd5-1d0e-491e-8140-9b1bd47dc813 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:06.865803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbaa55-f9bc-4456-a977-72f6468eead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:06.866667 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbaa55-f9bc-4456-a977-72f6468eead9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:06.866979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbaa55-f9bc-4456-a977-72f6468eead9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:06.867256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbaa55-f9bc-4456-a977-72f6468eead9 None admin] PATH_INFO: `/v3/groups/1bb905d2b42341d6a21106d8c2d3b728` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:06.868515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbaa55-f9bc-4456-a977-72f6468eead9 None admin] RBAC: Authorizing `identity:get_group(group_id=1bb905d2b42341d6a21106d8c2d3b728)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:06.869956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbaa55-f9bc-4456-a977-72f6468eead9 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:06.871734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 228/454] 10.222.0.23 () {56 vars in 1180 bytes} [Mon May 23 15:37:06 2022] GET /identity/v3/groups/1bb905d2b42341d6a21106d8c2d3b728 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:06.877424 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a86e9-b398-4db5-a3e3-57221c96d3b2 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:06.946434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:06.980069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a86e9-b398-4db5-a3e3-57221c96d3b2 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:07.014923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3876a0b-5fda-4797-ac15-06327b7f8f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:07.015575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3876a0b-5fda-4797-ac15-06327b7f8f0a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:07.015857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3876a0b-5fda-4797-ac15-06327b7f8f0a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:07.016131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3876a0b-5fda-4797-ac15-06327b7f8f0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:07.016994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3876a0b-5fda-4797-ac15-06327b7f8f0a None admin] RBAC: Adding query filter params (group.id=1bb905d2b42341d6a21106d8c2d3b728, role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=6e65bbb7bace4fb6a7f71d68e850edbd) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:07.017309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3876a0b-5fda-4797-ac15-06327b7f8f0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:07.018725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3876a0b-5fda-4797-ac15-06327b7f8f0a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:07.024730 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 227/455] 10.222.0.23 () {56 vars in 1376 bytes} [Mon May 23 15:37:06 2022] GET /identity/v3/role_assignments?group.id=1bb905d2b42341d6a21106d8c2d3b728&scope.project.id=6e65bbb7bace4fb6a7f71d68e850edbd&role.id=04c11dea81274a69974ad5c639f2c60b => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:09.599483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0021ae4-e083-48b5-8559-ad5bdea8f69a None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:09.600497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0021ae4-e083-48b5-8559-ad5bdea8f69a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:09.601009 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0021ae4-e083-48b5-8559-ad5bdea8f69a None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:09.602796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 229/456] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:09.608834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc51d83-0271-4478-a429-4691c27ca7b3 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:09.609633 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc51d83-0271-4478-a429-4691c27ca7b3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:09.610130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc51d83-0271-4478-a429-4691c27ca7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:09.633396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-adc51d83-0271-4478-a429-4691c27ca7b3 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:09.635187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-adc51d83-0271-4478-a429-4691c27ca7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:09.718706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 228/457] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:09.752329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0021ae4-e083-48b5-8559-ad5bdea8f69a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:09.878391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:09.909720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0021ae4-e083-48b5-8559-ad5bdea8f69a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:09.954643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a48049-3cf0-471d-ba26-f51fe3f278ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:09.955864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a48049-3cf0-471d-ba26-f51fe3f278ac None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:09.956347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a48049-3cf0-471d-ba26-f51fe3f278ac None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:09.956848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a48049-3cf0-471d-ba26-f51fe3f278ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:09.974788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a48049-3cf0-471d-ba26-f51fe3f278ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:09.976628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a48049-3cf0-471d-ba26-f51fe3f278ac None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-66a48049-3cf0-471d-ba26-f51fe3f278ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:10.002986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:10.003884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:10.004731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:10.004731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:10.004731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:10.004731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:10.006982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 230/458] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:10.012807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc51d83-0271-4478-a429-4691c27ca7b3 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:10.101827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:10.143976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc51d83-0271-4478-a429-4691c27ca7b3 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:10.159998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92cd79-b62a-4b2a-af1c-0e6cdc9bd223 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.160636 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92cd79-b62a-4b2a-af1c-0e6cdc9bd223 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.161137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92cd79-b62a-4b2a-af1c-0e6cdc9bd223 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.162934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 231/459] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:10.164192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2f066-9328-4061-bb6e-300f61b1e46b None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.168585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2f066-9328-4061-bb6e-300f61b1e46b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.169093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2f066-9328-4061-bb6e-300f61b1e46b None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.171116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 232/460] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:10.172572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f81cb-f92c-4bbc-8401-31231b2b39c9 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.173057 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f81cb-f92c-4bbc-8401-31231b2b39c9 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.173526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f81cb-f92c-4bbc-8401-31231b2b39c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.190499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89023191-0a9a-4d55-903a-8dcf1fee4be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:10.191173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89023191-0a9a-4d55-903a-8dcf1fee4be6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.191457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89023191-0a9a-4d55-903a-8dcf1fee4be6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.191733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89023191-0a9a-4d55-903a-8dcf1fee4be6 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.192497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89023191-0a9a-4d55-903a-8dcf1fee4be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:10.192805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89023191-0a9a-4d55-903a-8dcf1fee4be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:10.194189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89023191-0a9a-4d55-903a-8dcf1fee4be6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:10.196026 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-4e1f81cb-f92c-4bbc-8401-31231b2b39c9 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:10.197378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4e1f81cb-f92c-4bbc-8401-31231b2b39c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:10.206712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 229/461] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:10.207971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4a68e-9588-4f19-a245-2c3cdc7b9542 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.208262 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4a68e-9588-4f19-a245-2c3cdc7b9542 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.212686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4a68e-9588-4f19-a245-2c3cdc7b9542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.229351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a0c4a68e-9588-4f19-a245-2c3cdc7b9542 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:10.230543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a0c4a68e-9588-4f19-a245-2c3cdc7b9542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:10.271316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 233/462] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:10.272232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f81cb-f92c-4bbc-8401-31231b2b39c9 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:10.312861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 230/463] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:10.314087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4a68e-9588-4f19-a245-2c3cdc7b9542 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:10.370837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:10.392751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:10.410284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f81cb-f92c-4bbc-8401-31231b2b39c9 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:10.420243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4a68e-9588-4f19-a245-2c3cdc7b9542 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:10.448601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1a723-1183-4d1a-85f5-15101daadd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:10.449311 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a723-1183-4d1a-85f5-15101daadd82 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.449674 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a723-1183-4d1a-85f5-15101daadd82 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.450047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a723-1183-4d1a-85f5-15101daadd82 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.450808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1a723-1183-4d1a-85f5-15101daadd82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:10.453158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1a723-1183-4d1a-85f5-15101daadd82 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:10.457721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94378fea-7f29-4033-9ea8-daefe727eb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:10.458683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94378fea-7f29-4033-9ea8-daefe727eb00 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.459131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94378fea-7f29-4033-9ea8-daefe727eb00 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.459549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94378fea-7f29-4033-9ea8-daefe727eb00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.470542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94378fea-7f29-4033-9ea8-daefe727eb00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:10.472153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94378fea-7f29-4033-9ea8-daefe727eb00 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-94378fea-7f29-4033-9ea8-daefe727eb00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:10.490651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:10.491527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:10.492354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:10.492354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:10.492354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:10.492354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:10.492354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 231/464] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:10.494034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94378fea-7f29-4033-9ea8-daefe727eb00 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:10.545811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 234/465] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:37:10 2022] POST /identity/v3/users => generated 246 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:10.549520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1a723-1183-4d1a-85f5-15101daadd82 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:10.604812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94378fea-7f29-4033-9ea8-daefe727eb00 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:10.608725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:10.650417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022da06-d7c0-413c-a9c3-bd4bf93eaa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:10.651353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022da06-d7c0-413c-a9c3-bd4bf93eaa60 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.651644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022da06-d7c0-413c-a9c3-bd4bf93eaa60 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.651920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022da06-d7c0-413c-a9c3-bd4bf93eaa60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.667262 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022da06-d7c0-413c-a9c3-bd4bf93eaa60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:10.668675 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022da06-d7c0-413c-a9c3-bd4bf93eaa60 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-f022da06-d7c0-413c-a9c3-bd4bf93eaa60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:10.693689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:10.694782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:10.695827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:10.695827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:10.695827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:10.695827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:10.700694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 232/466] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:10.703238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c644b-c3b1-4baa-9486-52b04d9a6711 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.703511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c644b-c3b1-4baa-9486-52b04d9a6711 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.703789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c644b-c3b1-4baa-9486-52b04d9a6711 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.705579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 233/467] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:10.706437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c644b-c3b1-4baa-9486-52b04d9a6711 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:10.714855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1a723-1183-4d1a-85f5-15101daadd82 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:10.733870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:10.753635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29ffb3-5910-4743-b48d-fe6d1d4c88cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:10.754182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29ffb3-5910-4743-b48d-fe6d1d4c88cb None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.755358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29ffb3-5910-4743-b48d-fe6d1d4c88cb None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.755358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29ffb3-5910-4743-b48d-fe6d1d4c88cb None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.755521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29ffb3-5910-4743-b48d-fe6d1d4c88cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:10.755745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29ffb3-5910-4743-b48d-fe6d1d4c88cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:10.759599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29ffb3-5910-4743-b48d-fe6d1d4c88cb None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:10.771305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 235/468] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:10.772735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd216be-4447-4f4a-97d4-a477173e005f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.772735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd216be-4447-4f4a-97d4-a477173e005f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.772989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd216be-4447-4f4a-97d4-a477173e005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.794014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0cd216be-4447-4f4a-97d4-a477173e005f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:10.795224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0cd216be-4447-4f4a-97d4-a477173e005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:10.838756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c644b-c3b1-4baa-9486-52b04d9a6711 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:10.875047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 236/469] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:10.876183 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd216be-4447-4f4a-97d4-a477173e005f None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:10.883780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a1b64-093e-4766-a0bd-b53a3fcd0578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:10.884484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a1b64-093e-4766-a0bd-b53a3fcd0578 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.884771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a1b64-093e-4766-a0bd-b53a3fcd0578 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.885633 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a1b64-093e-4766-a0bd-b53a3fcd0578 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.886047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a1b64-093e-4766-a0bd-b53a3fcd0578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:10.886356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a1b64-093e-4766-a0bd-b53a3fcd0578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:10.888065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a1b64-093e-4766-a0bd-b53a3fcd0578 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:10.906578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 234/470] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:10.907962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61508f7-6447-475f-b871-48f9884e786f None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:10.908212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61508f7-6447-475f-b871-48f9884e786f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:10.912467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61508f7-6447-475f-b871-48f9884e786f None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:10.913782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 235/471] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:10.914866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61508f7-6447-475f-b871-48f9884e786f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:10.934503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:10.990842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd216be-4447-4f4a-97d4-a477173e005f None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:11.002332 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:11.025283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61508f7-6447-475f-b871-48f9884e786f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:11.030513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946d76b-1cdc-4616-a402-74f240e12705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:11.031198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946d76b-1cdc-4616-a402-74f240e12705 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:11.031482 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946d76b-1cdc-4616-a402-74f240e12705 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:11.031758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946d76b-1cdc-4616-a402-74f240e12705 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:11.032505 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946d76b-1cdc-4616-a402-74f240e12705 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:11.033984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946d76b-1cdc-4616-a402-74f240e12705 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:11.060005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62486c4-d024-4692-8865-d3ebd73bbc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:11.060827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62486c4-d024-4692-8865-d3ebd73bbc4a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:11.061647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62486c4-d024-4692-8865-d3ebd73bbc4a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:11.061922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62486c4-d024-4692-8865-d3ebd73bbc4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:11.071831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62486c4-d024-4692-8865-d3ebd73bbc4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:11.073324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62486c4-d024-4692-8865-d3ebd73bbc4a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-e62486c4-d024-4692-8865-d3ebd73bbc4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:11.090407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:11.091287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:11.092131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:11.092131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:11.092131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:11.092131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:11.092131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 236/472] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:11.092976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32771c-2f32-42f1-9458-73060af6198c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:11.093145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32771c-2f32-42f1-9458-73060af6198c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:11.093429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32771c-2f32-42f1-9458-73060af6198c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:11.110662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6c32771c-2f32-42f1-9458-73060af6198c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:11.111878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6c32771c-2f32-42f1-9458-73060af6198c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:11.116842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 237/473] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:37:10 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:11.117829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946d76b-1cdc-4616-a402-74f240e12705 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:11.207075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:11.214071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 237/474] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:11.214963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32771c-2f32-42f1-9458-73060af6198c None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:11.260948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946d76b-1cdc-4616-a402-74f240e12705 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:11.308663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedd914-8626-40ca-a5ef-56a082a96a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:11.309360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedd914-8626-40ca-a5ef-56a082a96a98 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:11.309646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedd914-8626-40ca-a5ef-56a082a96a98 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:11.309984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedd914-8626-40ca-a5ef-56a082a96a98 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:11.311042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedd914-8626-40ca-a5ef-56a082a96a98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:11.313248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedd914-8626-40ca-a5ef-56a082a96a98 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:11.351058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32771c-2f32-42f1-9458-73060af6198c None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:11.390146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44251436-572d-4871-b3aa-97d30e00927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:11.390828 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44251436-572d-4871-b3aa-97d30e00927f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:11.391124 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44251436-572d-4871-b3aa-97d30e00927f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:11.391393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44251436-572d-4871-b3aa-97d30e00927f None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:11.392131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44251436-572d-4871-b3aa-97d30e00927f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:11.392513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44251436-572d-4871-b3aa-97d30e00927f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:11.393893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44251436-572d-4871-b3aa-97d30e00927f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:11.403626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 238/475] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:37:11 2022] POST /identity/v3/users => generated 248 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:11.406003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedd914-8626-40ca-a5ef-56a082a96a98 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:11.410649 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 238/476] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:11.414895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44251436-572d-4871-b3aa-97d30e00927f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:11.432315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:11.485738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:11.560845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44251436-572d-4871-b3aa-97d30e00927f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:11.562951 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedd914-8626-40ca-a5ef-56a082a96a98 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:11.632353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500761e4-aa6d-42f6-a9d4-136de48e048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:11.633294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ddfed-ec99-4b19-b5df-f2472fd09835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:11.634154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500761e4-aa6d-42f6-a9d4-136de48e048a None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:11.635241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ddfed-ec99-4b19-b5df-f2472fd09835 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:11.635672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500761e4-aa6d-42f6-a9d4-136de48e048a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:11.636109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500761e4-aa6d-42f6-a9d4-136de48e048a None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:11.636574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ddfed-ec99-4b19-b5df-f2472fd09835 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:11.637462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500761e4-aa6d-42f6-a9d4-136de48e048a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:11.637888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ddfed-ec99-4b19-b5df-f2472fd09835 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:11.638788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ddfed-ec99-4b19-b5df-f2472fd09835 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:11.641020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500761e4-aa6d-42f6-a9d4-136de48e048a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:11.642566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ddfed-ec99-4b19-b5df-f2472fd09835 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:11.687139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.sql.core [None req-bf0ddfed-ec99-4b19-b5df-f2472fd09835 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 15:37:11.687139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 15:37:11.687139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [parameters: {'id': '5ace4fba042f4277be15605ea381f7c4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 15:37:11.687139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87073) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-bf0ddfed-ec99-4b19-b5df-f2472fd09835 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application conn.query(q) May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application result.read() May 23 15:37:11.704619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return ref.to_dict() May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:37:11.705672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application next(self.gen) May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application yield resource May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application next(self.gen) May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self.session.rollback() May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self.force_reraise() May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise self.value May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application session.commit() May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 15:37:11.706794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._prepare_impl() May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self.session.flush() May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._flush(objects) May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application compat.raise_( May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application flush_context.execute() May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rec.execute(self) May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 15:37:11.707886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:37:11.708982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application conn.query(q) May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application result.read() May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:37:11.709882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [parameters: {'id': '5ace4fba042f4277be15605ea381f7c4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:11.710666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 15:37:11.711560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:11.715446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 239/477] 10.222.0.23 () {60 vars in 1131 bytes} [Mon May 23 15:37:11 2022] POST /identity/v3/roles => generated 144 bytes in 310 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:11.721507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ddfed-ec99-4b19-b5df-f2472fd09835 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:11.763002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 239/478] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:37:11 2022] POST /identity/v3/users => generated 249 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:11.819844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:11.839737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ddfed-ec99-4b19-b5df-f2472fd09835 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:11.879363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1544d-cc52-4a46-8627-0e52a77a1df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:11.880210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1544d-cc52-4a46-8627-0e52a77a1df9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:11.880529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1544d-cc52-4a46-8627-0e52a77a1df9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:11.880810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1544d-cc52-4a46-8627-0e52a77a1df9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:11.894683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1544d-cc52-4a46-8627-0e52a77a1df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-06b1544d-cc52-4a46-8627-0e52a77a1df9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:11.902940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:11.903803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:11.904649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 240/479] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:11.910595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500761e4-aa6d-42f6-a9d4-136de48e048a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:12.039491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500761e4-aa6d-42f6-a9d4-136de48e048a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:12.059991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:12.090575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827d764-11fc-4c36-a33b-a7794e294a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:12.091271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827d764-11fc-4c36-a33b-a7794e294a9e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:12.091557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827d764-11fc-4c36-a33b-a7794e294a9e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:12.091836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827d764-11fc-4c36-a33b-a7794e294a9e None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:12.092668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827d764-11fc-4c36-a33b-a7794e294a9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:12.092979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827d764-11fc-4c36-a33b-a7794e294a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:12.094366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827d764-11fc-4c36-a33b-a7794e294a9e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:12.107950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 240/480] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:13.118003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d636d0-1dd5-4e54-b0cb-869e4a123af1 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:13.118931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d636d0-1dd5-4e54-b0cb-869e4a123af1 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:13.119371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d636d0-1dd5-4e54-b0cb-869e4a123af1 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:13.121165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 241/481] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:13.135346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c42e78-2fb7-4d9f-b21e-90fd8fcd2f1d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:13.135346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c42e78-2fb7-4d9f-b21e-90fd8fcd2f1d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:13.135346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c42e78-2fb7-4d9f-b21e-90fd8fcd2f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:13.153311 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c2c42e78-2fb7-4d9f-b21e-90fd8fcd2f1d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:13.154749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c2c42e78-2fb7-4d9f-b21e-90fd8fcd2f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:13.246520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 241/482] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:37:13.282641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d636d0-1dd5-4e54-b0cb-869e4a123af1 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:13.438159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d636d0-1dd5-4e54-b0cb-869e4a123af1 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:13.491020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57890972-3666-4f51-857e-7d9f81f9a978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:13.491839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57890972-3666-4f51-857e-7d9f81f9a978 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:13.492121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57890972-3666-4f51-857e-7d9f81f9a978 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:13.492417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57890972-3666-4f51-857e-7d9f81f9a978 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:13.503647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57890972-3666-4f51-857e-7d9f81f9a978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-57890972-3666-4f51-857e-7d9f81f9a978 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:13.511659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:13.512573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:13.513371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:13.518005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 242/483] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:13.522672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c42e78-2fb7-4d9f-b21e-90fd8fcd2f1d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:13.658230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829e44f-a967-4188-a1ca-717bfb48b7cb None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:13.658592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829e44f-a967-4188-a1ca-717bfb48b7cb None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:13.658796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829e44f-a967-4188-a1ca-717bfb48b7cb None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:13.660462 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 243/484] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:37:13.665158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a5c5-cb0d-419e-8b13-d52c233eeff1 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:13.665450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a5c5-cb0d-419e-8b13-d52c233eeff1 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:13.665733 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a5c5-cb0d-419e-8b13-d52c233eeff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:13.684038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c42e78-2fb7-4d9f-b21e-90fd8fcd2f1d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:13.684691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:13.692198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:13.697720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-25c1a5c5-cb0d-419e-8b13-d52c233eeff1 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:13.698909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-25c1a5c5-cb0d-419e-8b13-d52c233eeff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:13.726131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb47bfa-438f-4ae9-aa39-d124b433ace4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:13.726809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb47bfa-438f-4ae9-aa39-d124b433ace4 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:13.727091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb47bfa-438f-4ae9-aa39-d124b433ace4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:13.727359 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb47bfa-438f-4ae9-aa39-d124b433ace4 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:13.728170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb47bfa-438f-4ae9-aa39-d124b433ace4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:13.728442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb47bfa-438f-4ae9-aa39-d124b433ace4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:13.732102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb47bfa-438f-4ae9-aa39-d124b433ace4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:13.746596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 242/485] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:13.751372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb47bfa-438f-4ae9-aa39-d124b433ace4 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:13.800318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 244/486] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:37:13.839324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1a5c5-cb0d-419e-8b13-d52c233eeff1 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:13.937654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb47bfa-438f-4ae9-aa39-d124b433ace4 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:13.976221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdb8b7-e71c-4fca-b2e7-c95559719bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:13.977065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdb8b7-e71c-4fca-b2e7-c95559719bd5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:13.977350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdb8b7-e71c-4fca-b2e7-c95559719bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:13.977624 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdb8b7-e71c-4fca-b2e7-c95559719bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:13.989207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdb8b7-e71c-4fca-b2e7-c95559719bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:13.990661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdb8b7-e71c-4fca-b2e7-c95559719bd5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:13.994891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1a5c5-cb0d-419e-8b13-d52c233eeff1 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-befdb8b7-e71c-4fca-b2e7-c95559719bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:14.018030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:14.018929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:14.019751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:14.019751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:14.019751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:14.019751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:14.027121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:14.028739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 243/487] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:14.034612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ffa2f-84c7-46bb-99ef-fa6de3246028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:14.035438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ffa2f-84c7-46bb-99ef-fa6de3246028 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.035721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ffa2f-84c7-46bb-99ef-fa6de3246028 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.041660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ffa2f-84c7-46bb-99ef-fa6de3246028 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.041783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdb8b7-e71c-4fca-b2e7-c95559719bd5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:14.063189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ffa2f-84c7-46bb-99ef-fa6de3246028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-3c7ffa2f-84c7-46bb-99ef-fa6de3246028 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:14.071312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:14.072258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:14.073080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:14.084489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 245/488] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:14.093952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ffa2f-84c7-46bb-99ef-fa6de3246028 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:14.107316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:14.167364 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdb8b7-e71c-4fca-b2e7-c95559719bd5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:14.232453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a229c-4d4f-4743-bcdb-09a37d79e108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:14.233500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a229c-4d4f-4743-bcdb-09a37d79e108 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.233980 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a229c-4d4f-4743-bcdb-09a37d79e108 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.234468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a229c-4d4f-4743-bcdb-09a37d79e108 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.236427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a229c-4d4f-4743-bcdb-09a37d79e108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:14.236938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a229c-4d4f-4743-bcdb-09a37d79e108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:14.239202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a229c-4d4f-4743-bcdb-09a37d79e108 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:14.260391 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 244/489] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:14.261848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f235061-478e-4c64-babc-ab2292c34af7 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.262140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f235061-478e-4c64-babc-ab2292c34af7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.262415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f235061-478e-4c64-babc-ab2292c34af7 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.263790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 245/490] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:14.266353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f235061-478e-4c64-babc-ab2292c34af7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:14.301551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ffa2f-84c7-46bb-99ef-fa6de3246028 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:14.338291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65a45e-a196-4ca0-afbf-19eb607608a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:14.338963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65a45e-a196-4ca0-afbf-19eb607608a5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.339248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65a45e-a196-4ca0-afbf-19eb607608a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.339527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65a45e-a196-4ca0-afbf-19eb607608a5 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.340425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65a45e-a196-4ca0-afbf-19eb607608a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:14.340741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65a45e-a196-4ca0-afbf-19eb607608a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:14.342136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65a45e-a196-4ca0-afbf-19eb607608a5 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:14.351169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 246/491] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:14.352655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f637d10-e0e1-42bf-8ba2-d4eb5f2bc756 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.352945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f637d10-e0e1-42bf-8ba2-d4eb5f2bc756 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.353225 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f637d10-e0e1-42bf-8ba2-d4eb5f2bc756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.356847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:14.371020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-9f637d10-e0e1-42bf-8ba2-d4eb5f2bc756 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:14.372345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-9f637d10-e0e1-42bf-8ba2-d4eb5f2bc756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:14.379127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f235061-478e-4c64-babc-ab2292c34af7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:14.416467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835457e0-126b-4487-a0f2-be4fd98462a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:14.417684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835457e0-126b-4487-a0f2-be4fd98462a2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.418177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835457e0-126b-4487-a0f2-be4fd98462a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.418741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835457e0-126b-4487-a0f2-be4fd98462a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.426331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:14.427809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835457e0-126b-4487-a0f2-be4fd98462a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:14.429731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835457e0-126b-4487-a0f2-be4fd98462a2 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-835457e0-126b-4487-a0f2-be4fd98462a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:14.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:14.448757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:14.449650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:14.449650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:14.449650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:14.449650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:14.449650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:14.449650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:14.449650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:14.449650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:14.449650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:14.449650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:14.452619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 246/493] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 23 15:37:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:14.452730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 247/493] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:14.453728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f637d10-e0e1-42bf-8ba2-d4eb5f2bc756 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:14.458486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835457e0-126b-4487-a0f2-be4fd98462a2 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:14.568020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835457e0-126b-4487-a0f2-be4fd98462a2 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:14.568221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f637d10-e0e1-42bf-8ba2-d4eb5f2bc756 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:14.602897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b235b-92f8-4b83-bd7b-6ea877007c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:14.603631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b235b-92f8-4b83-bd7b-6ea877007c33 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.603913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b235b-92f8-4b83-bd7b-6ea877007c33 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.604189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b235b-92f8-4b83-bd7b-6ea877007c33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.604779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b52573-18a8-44ff-b480-42de28e1c564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:14.605660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b52573-18a8-44ff-b480-42de28e1c564 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.606255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b52573-18a8-44ff-b480-42de28e1c564 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.606861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b52573-18a8-44ff-b480-42de28e1c564 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.607903 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b52573-18a8-44ff-b480-42de28e1c564 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:14.608971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b52573-18a8-44ff-b480-42de28e1c564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:14.610611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b52573-18a8-44ff-b480-42de28e1c564 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:14.615314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b235b-92f8-4b83-bd7b-6ea877007c33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:14.616808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b235b-92f8-4b83-bd7b-6ea877007c33 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-b17b235b-92f8-4b83-bd7b-6ea877007c33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:14.635614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:14.636541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:14.637382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:14.637382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:14.637382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:14.637382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:14.637382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 248/494] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:14.638057 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b235b-92f8-4b83-bd7b-6ea877007c33 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:14.641789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:14.659419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 247/495] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 23 15:37:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:14.659419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b52573-18a8-44ff-b480-42de28e1c564 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:14.680695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:14.751672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b235b-92f8-4b83-bd7b-6ea877007c33 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:14.758208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b52573-18a8-44ff-b480-42de28e1c564 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:14.787507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539a91a-684a-4201-a367-d595e0b3676e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:14.788303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539a91a-684a-4201-a367-d595e0b3676e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.788630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539a91a-684a-4201-a367-d595e0b3676e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.788913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539a91a-684a-4201-a367-d595e0b3676e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.793509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba328a51-fe58-4c4e-b50c-570d9a560707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:14.794379 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba328a51-fe58-4c4e-b50c-570d9a560707 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.794735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba328a51-fe58-4c4e-b50c-570d9a560707 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.795154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba328a51-fe58-4c4e-b50c-570d9a560707 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.796499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba328a51-fe58-4c4e-b50c-570d9a560707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:14.796810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba328a51-fe58-4c4e-b50c-570d9a560707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:14.798459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba328a51-fe58-4c4e-b50c-570d9a560707 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:14.800495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539a91a-684a-4201-a367-d595e0b3676e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-6539a91a-684a-4201-a367-d595e0b3676e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:14.808770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:14.809669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:14.816980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 249/496] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:14.817645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 248/497] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:14.817731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539a91a-684a-4201-a367-d595e0b3676e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:14.817798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c60e2-3bb0-4a30-959d-f4c57819fd5e None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.817798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c60e2-3bb0-4a30-959d-f4c57819fd5e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.817798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c60e2-3bb0-4a30-959d-f4c57819fd5e None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.817798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 249/498] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:14.817798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c60e2-3bb0-4a30-959d-f4c57819fd5e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:14.880706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:14.923245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:14.928125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c60e2-3bb0-4a30-959d-f4c57819fd5e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:14.929683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539a91a-684a-4201-a367-d595e0b3676e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:14.965198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808443d-23ec-42df-aa5e-1c83028feaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:14.965999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808443d-23ec-42df-aa5e-1c83028feaef None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.966283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808443d-23ec-42df-aa5e-1c83028feaef None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.966556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808443d-23ec-42df-aa5e-1c83028feaef None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.967122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa48fb-86c1-43c7-ac93-ce2b2c3fd1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:14.968307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa48fb-86c1-43c7-ac93-ce2b2c3fd1e2 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:14.968634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa48fb-86c1-43c7-ac93-ce2b2c3fd1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:14.968917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa48fb-86c1-43c7-ac93-ce2b2c3fd1e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:14.975947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808443d-23ec-42df-aa5e-1c83028feaef None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:14.977851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808443d-23ec-42df-aa5e-1c83028feaef None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-2808443d-23ec-42df-aa5e-1c83028feaef None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:14.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:14.993898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:14.994830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 250/499] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:14.995411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808443d-23ec-42df-aa5e-1c83028feaef None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.042283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa48fb-86c1-43c7-ac93-ce2b2c3fd1e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.043755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa48fb-86c1-43c7-ac93-ce2b2c3fd1e2 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.098604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808443d-23ec-42df-aa5e-1c83028feaef None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.116837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:15.133521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b20ef7-9594-480b-ac37-0304c12e54c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.134125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20ef7-9594-480b-ac37-0304c12e54c9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.134405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20ef7-9594-480b-ac37-0304c12e54c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.134676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20ef7-9594-480b-ac37-0304c12e54c9 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.135685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b20ef7-9594-480b-ac37-0304c12e54c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:15.135989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b20ef7-9594-480b-ac37-0304c12e54c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.137386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b20ef7-9594-480b-ac37-0304c12e54c9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.141971 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:15.147886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 251/500] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:15.153123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 250/501] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:15.153263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f71bd-df8d-46da-b51d-bb833b6eb3e6 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.153354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa48fb-86c1-43c7-ac93-ce2b2c3fd1e2 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.153436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f71bd-df8d-46da-b51d-bb833b6eb3e6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.153436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f71bd-df8d-46da-b51d-bb833b6eb3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.165597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-769f71bd-df8d-46da-b51d-bb833b6eb3e6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:15.166781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-769f71bd-df8d-46da-b51d-bb833b6eb3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:15.232150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 252/502] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:15.233869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3afe7-475a-4e70-8657-4f5f92beef44 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.234155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3afe7-475a-4e70-8657-4f5f92beef44 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.234423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3afe7-475a-4e70-8657-4f5f92beef44 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.235811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 253/503] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:15.237228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3afe7-475a-4e70-8657-4f5f92beef44 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.253823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa48fb-86c1-43c7-ac93-ce2b2c3fd1e2 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.292592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cde86-5a6e-44ee-bad2-ae5f1d073ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.293269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cde86-5a6e-44ee-bad2-ae5f1d073ef4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.293559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cde86-5a6e-44ee-bad2-ae5f1d073ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.293887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cde86-5a6e-44ee-bad2-ae5f1d073ef4 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.294753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cde86-5a6e-44ee-bad2-ae5f1d073ef4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:15.295064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cde86-5a6e-44ee-bad2-ae5f1d073ef4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.296586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cde86-5a6e-44ee-bad2-ae5f1d073ef4 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.332141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 251/504] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:15.333594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cde86-5a6e-44ee-bad2-ae5f1d073ef4 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.338593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3afe7-475a-4e70-8657-4f5f92beef44 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.349842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:15.373567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d748d30-1270-4a07-a247-32c6acf6fb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.374357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d748d30-1270-4a07-a247-32c6acf6fb26 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.374639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d748d30-1270-4a07-a247-32c6acf6fb26 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.374913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d748d30-1270-4a07-a247-32c6acf6fb26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.386037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d748d30-1270-4a07-a247-32c6acf6fb26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.387432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d748d30-1270-4a07-a247-32c6acf6fb26 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.390840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-0d748d30-1270-4a07-a247-32c6acf6fb26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:15.405537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:15.406373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:15.407200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:15.407200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:15.407200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:15.407200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:15.407200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 254/505] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:15.408029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafbbcd-0380-499c-8e16-976e4d6cfdcd None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.408308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafbbcd-0380-499c-8e16-976e4d6cfdcd None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.408602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafbbcd-0380-499c-8e16-976e4d6cfdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.424591 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9aafbbcd-0380-499c-8e16-976e4d6cfdcd None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:15.425755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9aafbbcd-0380-499c-8e16-976e4d6cfdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:15.435024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cde86-5a6e-44ee-bad2-ae5f1d073ef4 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.467174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bce7a-9e8e-45b9-853d-b28a5ab155a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.468604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bce7a-9e8e-45b9-853d-b28a5ab155a4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.468883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bce7a-9e8e-45b9-853d-b28a5ab155a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.469154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bce7a-9e8e-45b9-853d-b28a5ab155a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.479429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bce7a-9e8e-45b9-853d-b28a5ab155a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.480918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bce7a-9e8e-45b9-853d-b28a5ab155a4 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.490511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 255/506] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:15.491337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafbbcd-0380-499c-8e16-976e4d6cfdcd None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ed8bce7a-9e8e-45b9-853d-b28a5ab155a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:15.497650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:15.498610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:15.499504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:15.499504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:15.499504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:15.499504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:15.499504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 252/507] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:15.499766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bce7a-9e8e-45b9-853d-b28a5ab155a4 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.590035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:15.598844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bce7a-9e8e-45b9-853d-b28a5ab155a4 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.598983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafbbcd-0380-499c-8e16-976e4d6cfdcd None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.605678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:15.631362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c7155-fb80-4110-a12a-2642171e4b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.632007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c7155-fb80-4110-a12a-2642171e4b8b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.632135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4fbc4-85a6-4470-b19e-8cff21a3242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.632291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c7155-fb80-4110-a12a-2642171e4b8b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.632598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c7155-fb80-4110-a12a-2642171e4b8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.632877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4fbc4-85a6-4470-b19e-8cff21a3242d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.633164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4fbc4-85a6-4470-b19e-8cff21a3242d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.633466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4fbc4-85a6-4470-b19e-8cff21a3242d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.644420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4fbc4-85a6-4470-b19e-8cff21a3242d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-55b4fbc4-85a6-4470-b19e-8cff21a3242d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:15.651962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:15.652920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:15.653702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 256/508] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:15.654877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4fbc4-85a6-4470-b19e-8cff21a3242d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.704391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c7155-fb80-4110-a12a-2642171e4b8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.705868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c7155-fb80-4110-a12a-2642171e4b8b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.752022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4fbc4-85a6-4470-b19e-8cff21a3242d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.786298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d60ce4-87e8-44f5-ac61-0d56bf4e8a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.786948 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d60ce4-87e8-44f5-ac61-0d56bf4e8a7e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.787227 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d60ce4-87e8-44f5-ac61-0d56bf4e8a7e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.787509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d60ce4-87e8-44f5-ac61-0d56bf4e8a7e None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.788215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d60ce4-87e8-44f5-ac61-0d56bf4e8a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:15.788556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d60ce4-87e8-44f5-ac61-0d56bf4e8a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.789905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d60ce4-87e8-44f5-ac61-0d56bf4e8a7e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.801832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 257/509] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:15.802602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d60ce4-87e8-44f5-ac61-0d56bf4e8a7e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.807718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 253/510] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:15.807718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c7155-fb80-4110-a12a-2642171e4b8b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.814750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:15.819021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:15.903272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d60ce4-87e8-44f5-ac61-0d56bf4e8a7e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.909138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c7155-fb80-4110-a12a-2642171e4b8b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:15.937528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d506a-25b3-4da5-b0ce-d18d5c84e78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.938190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d506a-25b3-4da5-b0ce-d18d5c84e78a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.938463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d506a-25b3-4da5-b0ce-d18d5c84e78a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.938740 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d506a-25b3-4da5-b0ce-d18d5c84e78a None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.939456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d506a-25b3-4da5-b0ce-d18d5c84e78a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:15.939761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d506a-25b3-4da5-b0ce-d18d5c84e78a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.941151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d506a-25b3-4da5-b0ce-d18d5c84e78a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.942463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910d262-4309-4f1d-867b-66f75be5b38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:15.943254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d262-4309-4f1d-867b-66f75be5b38e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:15.943544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d262-4309-4f1d-867b-66f75be5b38e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:15.943825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d262-4309-4f1d-867b-66f75be5b38e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:15.953057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 258/511] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:15.953894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d506a-25b3-4da5-b0ce-d18d5c84e78a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:15.954562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910d262-4309-4f1d-867b-66f75be5b38e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:15.956022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910d262-4309-4f1d-867b-66f75be5b38e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-8910d262-4309-4f1d-867b-66f75be5b38e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:15.973231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:15.974222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:15.975053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:15.975053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:15.975053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:15.975053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:15.975053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 254/512] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:15.975320 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910d262-4309-4f1d-867b-66f75be5b38e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.028932 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.032323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.057350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d506a-25b3-4da5-b0ce-d18d5c84e78a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.072793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910d262-4309-4f1d-867b-66f75be5b38e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.091641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11805799-a399-40e5-8cab-c2f383a50717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.092480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11805799-a399-40e5-8cab-c2f383a50717 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.093272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11805799-a399-40e5-8cab-c2f383a50717 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.093553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11805799-a399-40e5-8cab-c2f383a50717 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.094294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11805799-a399-40e5-8cab-c2f383a50717 None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:16.094599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11805799-a399-40e5-8cab-c2f383a50717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.095953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11805799-a399-40e5-8cab-c2f383a50717 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.103733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 259/513] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:16.104537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11805799-a399-40e5-8cab-c2f383a50717 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.107817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556ea4d-20a4-44db-9902-0a585f968de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.108675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556ea4d-20a4-44db-9902-0a585f968de0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.108954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556ea4d-20a4-44db-9902-0a585f968de0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.109226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556ea4d-20a4-44db-9902-0a585f968de0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.117418 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556ea4d-20a4-44db-9902-0a585f968de0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.118853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556ea4d-20a4-44db-9902-0a585f968de0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-f556ea4d-20a4-44db-9902-0a585f968de0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:16.134210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:16.139017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 255/514] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:16.140456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556ea4d-20a4-44db-9902-0a585f968de0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.205145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11805799-a399-40e5-8cab-c2f383a50717 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.237338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556ea4d-20a4-44db-9902-0a585f968de0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.237613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036f854-6e18-4114-b2b5-98256df5feb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.238398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036f854-6e18-4114-b2b5-98256df5feb9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.238680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036f854-6e18-4114-b2b5-98256df5feb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.238949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036f854-6e18-4114-b2b5-98256df5feb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.240950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.249817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036f854-6e18-4114-b2b5-98256df5feb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.251285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036f854-6e18-4114-b2b5-98256df5feb9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.253858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-8036f854-6e18-4114-b2b5-98256df5feb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:16.269069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:16.269900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:16.270738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:16.270738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:16.270738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:16.270738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:16.270738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 260/515] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:16.271185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036f854-6e18-4114-b2b5-98256df5feb9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.275347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e4d0-e355-421c-872e-84a3f42f9885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.276142 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e4d0-e355-421c-872e-84a3f42f9885 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.281510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e4d0-e355-421c-872e-84a3f42f9885 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.281510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e4d0-e355-421c-872e-84a3f42f9885 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.284323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e4d0-e355-421c-872e-84a3f42f9885 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.285823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e4d0-e355-421c-872e-84a3f42f9885 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-f1d9e4d0-e355-421c-872e-84a3f42f9885 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:16.301945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:16.302926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:16.303758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:16.303758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:16.303758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:16.303758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:16.303758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:16.303758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:16.303758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:16.303758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:16.303758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 256/516] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:16.304524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e4d0-e355-421c-872e-84a3f42f9885 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.378158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036f854-6e18-4114-b2b5-98256df5feb9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.401721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e4d0-e355-421c-872e-84a3f42f9885 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.411438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13001341-378d-4ff1-baaf-e43749c6ea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.412102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13001341-378d-4ff1-baaf-e43749c6ea16 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.412419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13001341-378d-4ff1-baaf-e43749c6ea16 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.412702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13001341-378d-4ff1-baaf-e43749c6ea16 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.413439 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13001341-378d-4ff1-baaf-e43749c6ea16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:16.413749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13001341-378d-4ff1-baaf-e43749c6ea16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.415116 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13001341-378d-4ff1-baaf-e43749c6ea16 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.427326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 261/517] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:16.428108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13001341-378d-4ff1-baaf-e43749c6ea16 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.438790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b627fa-b756-4949-93b3-be64309324b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.439640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b627fa-b756-4949-93b3-be64309324b8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.439969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b627fa-b756-4949-93b3-be64309324b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.440273 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b627fa-b756-4949-93b3-be64309324b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.452126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b627fa-b756-4949-93b3-be64309324b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.453760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b627fa-b756-4949-93b3-be64309324b8 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-58b627fa-b756-4949-93b3-be64309324b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:16.472722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:16.473568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:16.474444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:16.474444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:16.474444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:16.474444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:16.474444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 257/518] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:16.477172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b627fa-b756-4949-93b3-be64309324b8 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.488667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.506778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.541936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13001341-378d-4ff1-baaf-e43749c6ea16 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.574350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ce8b-c320-4448-b247-0ae1245a29ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.575018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ce8b-c320-4448-b247-0ae1245a29ba None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.575120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b627fa-b756-4949-93b3-be64309324b8 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.575303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ce8b-c320-4448-b247-0ae1245a29ba None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.575587 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ce8b-c320-4448-b247-0ae1245a29ba None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.576478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ce8b-c320-4448-b247-0ae1245a29ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:16.576788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ce8b-c320-4448-b247-0ae1245a29ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.578231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ce8b-c320-4448-b247-0ae1245a29ba None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.595476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 262/519] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:16.596282 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ce8b-c320-4448-b247-0ae1245a29ba None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.610895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be25ac-e97e-4722-a49e-06e59d20cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.611554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be25ac-e97e-4722-a49e-06e59d20cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.611834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be25ac-e97e-4722-a49e-06e59d20cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.612114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be25ac-e97e-4722-a49e-06e59d20cbb7 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.612860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be25ac-e97e-4722-a49e-06e59d20cbb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:16.613165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be25ac-e97e-4722-a49e-06e59d20cbb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.614538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be25ac-e97e-4722-a49e-06e59d20cbb7 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.625652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 258/520] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:16.626247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be25ac-e97e-4722-a49e-06e59d20cbb7 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.692239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ce8b-c320-4448-b247-0ae1245a29ba None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.694436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.714776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.724473 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be25ac-e97e-4722-a49e-06e59d20cbb7 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.725940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330dce36-dfd6-40c4-beb6-0b9ebf530b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.726613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dce36-dfd6-40c4-beb6-0b9ebf530b9b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.726895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dce36-dfd6-40c4-beb6-0b9ebf530b9b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.727180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dce36-dfd6-40c4-beb6-0b9ebf530b9b None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.727967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330dce36-dfd6-40c4-beb6-0b9ebf530b9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:16.728276 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330dce36-dfd6-40c4-beb6-0b9ebf530b9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.729731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330dce36-dfd6-40c4-beb6-0b9ebf530b9b None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.746999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 263/521] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:16.747720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330dce36-dfd6-40c4-beb6-0b9ebf530b9b None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.759593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b18e3-e907-43e6-8dbb-1ead109511cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.760561 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b18e3-e907-43e6-8dbb-1ead109511cc None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.760846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b18e3-e907-43e6-8dbb-1ead109511cc None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.761119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b18e3-e907-43e6-8dbb-1ead109511cc None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.761822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b18e3-e907-43e6-8dbb-1ead109511cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.763256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b18e3-e907-43e6-8dbb-1ead109511cc None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.847813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 259/522] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:37:16 2022] POST /identity/v3/users => generated 247 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:16.848811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b18e3-e907-43e6-8dbb-1ead109511cc None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.849398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330dce36-dfd6-40c4-beb6-0b9ebf530b9b None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:16.888653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c35af-af46-4ef2-8d21-fc6c20422f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:16.889331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c35af-af46-4ef2-8d21-fc6c20422f47 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:16.889618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c35af-af46-4ef2-8d21-fc6c20422f47 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:16.889896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c35af-af46-4ef2-8d21-fc6c20422f47 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:16.890646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c35af-af46-4ef2-8d21-fc6c20422f47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:16.890973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c35af-af46-4ef2-8d21-fc6c20422f47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:16.892402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c35af-af46-4ef2-8d21-fc6c20422f47 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:16.908933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 264/523] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:16.908933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c35af-af46-4ef2-8d21-fc6c20422f47 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:16.936090 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:16.965300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b18e3-e907-43e6-8dbb-1ead109511cc None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.006782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac7b5f-8ddc-46d2-8280-c9619161874e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.007623 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac7b5f-8ddc-46d2-8280-c9619161874e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.008116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac7b5f-8ddc-46d2-8280-c9619161874e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.008501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac7b5f-8ddc-46d2-8280-c9619161874e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.020597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c35af-af46-4ef2-8d21-fc6c20422f47 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.025520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:17.063384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6607ea9-71e0-4e55-b5fb-b83cd109bc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.064219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6607ea9-71e0-4e55-b5fb-b83cd109bc59 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.064566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6607ea9-71e0-4e55-b5fb-b83cd109bc59 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.064848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6607ea9-71e0-4e55-b5fb-b83cd109bc59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.072754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6607ea9-71e0-4e55-b5fb-b83cd109bc59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.074212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6607ea9-71e0-4e55-b5fb-b83cd109bc59 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.088200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac7b5f-8ddc-46d2-8280-c9619161874e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.089739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac7b5f-8ddc-46d2-8280-c9619161874e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-e6607ea9-71e0-4e55-b5fb-b83cd109bc59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:17.091731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:17.093263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:17.094229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:17.094229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:17.094229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:17.094229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:17.094229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:17.094229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:17.094229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:17.094229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:17.094229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 265/524] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:17.094786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6607ea9-71e0-4e55-b5fb-b83cd109bc59 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.185777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:17.201305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 260/525] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:17.202249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac7b5f-8ddc-46d2-8280-c9619161874e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.203388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6607ea9-71e0-4e55-b5fb-b83cd109bc59 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.242731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d46b66-c456-463c-85a2-ce8c481a65b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.243477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d46b66-c456-463c-85a2-ce8c481a65b8 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.243669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d46b66-c456-463c-85a2-ce8c481a65b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.243941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d46b66-c456-463c-85a2-ce8c481a65b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.244807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d46b66-c456-463c-85a2-ce8c481a65b8 None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=b9129f0b362a4945a0eca7cc33571abd) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:17.245121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d46b66-c456-463c-85a2-ce8c481a65b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.246554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d46b66-c456-463c-85a2-ce8c481a65b8 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.252227 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 266/526] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:17 2022] GET /identity/v3/role_assignments?user.id=b9129f0b362a4945a0eca7cc33571abd&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:17.253104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:17.253214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d46b66-c456-463c-85a2-ce8c481a65b8 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.345766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac7b5f-8ddc-46d2-8280-c9619161874e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.385616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8afdb-85c8-41d3-b5e9-f078d0530480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.386463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8afdb-85c8-41d3-b5e9-f078d0530480 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.386748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8afdb-85c8-41d3-b5e9-f078d0530480 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.387026 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8afdb-85c8-41d3-b5e9-f078d0530480 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.390936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d46b66-c456-463c-85a2-ce8c481a65b8 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.396431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8afdb-85c8-41d3-b5e9-f078d0530480 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.397952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8afdb-85c8-41d3-b5e9-f078d0530480 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-85e8afdb-85c8-41d3-b5e9-f078d0530480 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:17.414496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:17.415874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:17.416812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 261/527] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:17.417384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8afdb-85c8-41d3-b5e9-f078d0530480 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.442300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba31c95-a135-4339-bcc5-09fc544496d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.443313 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba31c95-a135-4339-bcc5-09fc544496d6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.443601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba31c95-a135-4339-bcc5-09fc544496d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.444952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba31c95-a135-4339-bcc5-09fc544496d6 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.444952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba31c95-a135-4339-bcc5-09fc544496d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:17.445174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba31c95-a135-4339-bcc5-09fc544496d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.447847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba31c95-a135-4339-bcc5-09fc544496d6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.463232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 267/528] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:17.464072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba31c95-a135-4339-bcc5-09fc544496d6 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.481458 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:17.539632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8afdb-85c8-41d3-b5e9-f078d0530480 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.548294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:17.576065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a38c2-76ca-49a1-8e15-c23ddf7ddfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.576934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a38c2-76ca-49a1-8e15-c23ddf7ddfee None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.577223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a38c2-76ca-49a1-8e15-c23ddf7ddfee None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.577504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a38c2-76ca-49a1-8e15-c23ddf7ddfee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.588930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a38c2-76ca-49a1-8e15-c23ddf7ddfee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.590389 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a38c2-76ca-49a1-8e15-c23ddf7ddfee None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.598676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba31c95-a135-4339-bcc5-09fc544496d6 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-458a38c2-76ca-49a1-8e15-c23ddf7ddfee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:17.609028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:17.610076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:17.611119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:17.611119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:17.611119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:17.611119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:17.611119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 262/529] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:17.611583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a38c2-76ca-49a1-8e15-c23ddf7ddfee None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.640290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0c553-cd6a-4ff1-bde7-c2c6605ecc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.641647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0c553-cd6a-4ff1-bde7-c2c6605ecc95 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.642133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0c553-cd6a-4ff1-bde7-c2c6605ecc95 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.642611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0c553-cd6a-4ff1-bde7-c2c6605ecc95 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.643753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0c553-cd6a-4ff1-bde7-c2c6605ecc95 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:17.644379 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0c553-cd6a-4ff1-bde7-c2c6605ecc95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.646027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0c553-cd6a-4ff1-bde7-c2c6605ecc95 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.665488 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 268/530] 10.222.0.23 () {56 vars in 1139 bytes} [Mon May 23 15:37:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:17.666619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0c553-cd6a-4ff1-bde7-c2c6605ecc95 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.734581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a38c2-76ca-49a1-8e15-c23ddf7ddfee None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.760873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:17.779616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927ebc4-a089-41d3-9293-a1fa92fea516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.781100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927ebc4-a089-41d3-9293-a1fa92fea516 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.781530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927ebc4-a089-41d3-9293-a1fa92fea516 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.782081 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927ebc4-a089-41d3-9293-a1fa92fea516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.783440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927ebc4-a089-41d3-9293-a1fa92fea516 None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=0d30c8d1f883400ba397ea09d42a0aa8) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:17.784205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927ebc4-a089-41d3-9293-a1fa92fea516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.787385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927ebc4-a089-41d3-9293-a1fa92fea516 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.796322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 263/531] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:17 2022] GET /identity/v3/role_assignments?user.id=0d30c8d1f883400ba397ea09d42a0aa8&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:17.797648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927ebc4-a089-41d3-9293-a1fa92fea516 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.803171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0c553-cd6a-4ff1-bde7-c2c6605ecc95 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:17.834271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:17.841267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15031abd-561b-4e3f-84af-7a4d37abfe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:17.842135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15031abd-561b-4e3f-84af-7a4d37abfe22 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:17.842567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15031abd-561b-4e3f-84af-7a4d37abfe22 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:17.849225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15031abd-561b-4e3f-84af-7a4d37abfe22 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:17.849225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15031abd-561b-4e3f-84af-7a4d37abfe22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:17.849225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15031abd-561b-4e3f-84af-7a4d37abfe22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:17.849225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15031abd-561b-4e3f-84af-7a4d37abfe22 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:17.859016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 269/532] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:17.864741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15031abd-561b-4e3f-84af-7a4d37abfe22 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:17.979399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927ebc4-a089-41d3-9293-a1fa92fea516 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.049532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c9c7a-b7e7-4dc0-ad25-6ff16bcc9964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.050400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c9c7a-b7e7-4dc0-ad25-6ff16bcc9964 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.050689 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c9c7a-b7e7-4dc0-ad25-6ff16bcc9964 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.050973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c9c7a-b7e7-4dc0-ad25-6ff16bcc9964 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.062350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15031abd-561b-4e3f-84af-7a4d37abfe22 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.094480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:18.105216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc6cb8-a3c0-468b-bf97-b5ee7c9dc01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.106022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc6cb8-a3c0-468b-bf97-b5ee7c9dc01c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.106312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc6cb8-a3c0-468b-bf97-b5ee7c9dc01c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.112283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc6cb8-a3c0-468b-bf97-b5ee7c9dc01c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.115393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc6cb8-a3c0-468b-bf97-b5ee7c9dc01c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.116874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc6cb8-a3c0-468b-bf97-b5ee7c9dc01c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-effc6cb8-a3c0-468b-bf97-b5ee7c9dc01c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:18.138259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:18.139931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:18.140888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:18.140888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:18.140888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:18.140888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:18.140888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:18.140888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:18.140888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:18.140888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:18.140888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 270/533] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:18.147328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc6cb8-a3c0-468b-bf97-b5ee7c9dc01c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.181426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c9c7a-b7e7-4dc0-ad25-6ff16bcc9964 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.183083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c9c7a-b7e7-4dc0-ad25-6ff16bcc9964 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.220754 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:18.340779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc6cb8-a3c0-468b-bf97-b5ee7c9dc01c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.358931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 264/534] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:18.373407 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c9c7a-b7e7-4dc0-ad25-6ff16bcc9964 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.383848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0061c-8546-4cd5-993e-0768f4964dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.384556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0061c-8546-4cd5-993e-0768f4964dbd None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.384842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0061c-8546-4cd5-993e-0768f4964dbd None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.385388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0061c-8546-4cd5-993e-0768f4964dbd None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.386259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0061c-8546-4cd5-993e-0768f4964dbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:18.386571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0061c-8546-4cd5-993e-0768f4964dbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.387997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0061c-8546-4cd5-993e-0768f4964dbd None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.403795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 271/535] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:18.414860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0061c-8546-4cd5-993e-0768f4964dbd None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.523355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:18.553834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c9c7a-b7e7-4dc0-ad25-6ff16bcc9964 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.578071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:18.590255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdfcaa-0aac-4e33-ba5e-fa0ac4260022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.591119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdfcaa-0aac-4e33-ba5e-fa0ac4260022 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.591356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdfcaa-0aac-4e33-ba5e-fa0ac4260022 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.591635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdfcaa-0aac-4e33-ba5e-fa0ac4260022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.602937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdfcaa-0aac-4e33-ba5e-fa0ac4260022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.604456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdfcaa-0aac-4e33-ba5e-fa0ac4260022 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.620029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0061c-8546-4cd5-993e-0768f4964dbd None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-13fdfcaa-0aac-4e33-ba5e-fa0ac4260022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:18.622550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:18.623368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:18.624266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:18.624266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:18.624266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:18.624266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:18.624266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 265/536] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:18.629528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdfcaa-0aac-4e33-ba5e-fa0ac4260022 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.736427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdfcaa-0aac-4e33-ba5e-fa0ac4260022 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.741183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda493e6-1576-420f-b82e-5d2fc91b00a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.741877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda493e6-1576-420f-b82e-5d2fc91b00a9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.742164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda493e6-1576-420f-b82e-5d2fc91b00a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.742439 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda493e6-1576-420f-b82e-5d2fc91b00a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.743295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda493e6-1576-420f-b82e-5d2fc91b00a9 None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=142e6e44d6544f6a891ed3f913de74ae) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:18.743616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda493e6-1576-420f-b82e-5d2fc91b00a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.745111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda493e6-1576-420f-b82e-5d2fc91b00a9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.762717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 272/537] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:18 2022] GET /identity/v3/role_assignments?user.id=142e6e44d6544f6a891ed3f913de74ae&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 259 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:18.774324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8bac0-43f0-46db-9f4d-0f8d26fab5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:18.775022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8bac0-43f0-46db-9f4d-0f8d26fab5ca None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:18.775305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8bac0-43f0-46db-9f4d-0f8d26fab5ca None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:18.775583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8bac0-43f0-46db-9f4d-0f8d26fab5ca None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:18.776333 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8bac0-43f0-46db-9f4d-0f8d26fab5ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:18.776676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8bac0-43f0-46db-9f4d-0f8d26fab5ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:18.778072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8bac0-43f0-46db-9f4d-0f8d26fab5ca None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:18.796737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 266/538] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:18.798135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda493e6-1576-420f-b82e-5d2fc91b00a9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:18.953874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda493e6-1576-420f-b82e-5d2fc91b00a9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:18.971381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:19.003564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3b507-5160-4712-bcc7-7db8bebfa66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:19.004655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3b507-5160-4712-bcc7-7db8bebfa66e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.005079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3b507-5160-4712-bcc7-7db8bebfa66e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.005494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3b507-5160-4712-bcc7-7db8bebfa66e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.015312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3b507-5160-4712-bcc7-7db8bebfa66e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:19.016959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3b507-5160-4712-bcc7-7db8bebfa66e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-4cc3b507-5160-4712-bcc7-7db8bebfa66e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:19.044599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:19.047030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:19.049252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:19.049252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:19.049252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:19.049252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:19.049252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:19.049252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:19.049252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:19.049252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:19.052142 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 273/539] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:19.061002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8bac0-43f0-46db-9f4d-0f8d26fab5ca None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:19.117314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:19.194705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8bac0-43f0-46db-9f4d-0f8d26fab5ca None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:19.244319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bd3f2-7582-4824-b440-60cec13d74fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:19.250214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd3f2-7582-4824-b440-60cec13d74fd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.250677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd3f2-7582-4824-b440-60cec13d74fd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.250961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd3f2-7582-4824-b440-60cec13d74fd None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.251791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bd3f2-7582-4824-b440-60cec13d74fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:19.252104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bd3f2-7582-4824-b440-60cec13d74fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:19.253626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bd3f2-7582-4824-b440-60cec13d74fd None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:19.280637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 267/540] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:19.289438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3b507-5160-4712-bcc7-7db8bebfa66e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:19.431225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3b507-5160-4712-bcc7-7db8bebfa66e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:19.464401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eaf42-a800-42e3-a232-c63875a75e09 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.464401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eaf42-a800-42e3-a232-c63875a75e09 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.464401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eaf42-a800-42e3-a232-c63875a75e09 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.465075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 268/541] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:19.479560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3e2c8-cb67-4083-82fc-456717df051f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.479998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3e2c8-cb67-4083-82fc-456717df051f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.480134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d94b11-b748-4e7b-b942-fc4f52bfccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:19.480606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3e2c8-cb67-4083-82fc-456717df051f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.480821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d94b11-b748-4e7b-b942-fc4f52bfccd9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.481102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d94b11-b748-4e7b-b942-fc4f52bfccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.481380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d94b11-b748-4e7b-b942-fc4f52bfccd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.482242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d94b11-b748-4e7b-b942-fc4f52bfccd9 None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=a39f5189d63d467dbd914f553c261c01) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:19.482555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d94b11-b748-4e7b-b942-fc4f52bfccd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:19.484731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d94b11-b748-4e7b-b942-fc4f52bfccd9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:19.497755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 274/542] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:19 2022] GET /identity/v3/role_assignments?user.id=a39f5189d63d467dbd914f553c261c01&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:19.501579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-7be3e2c8-cb67-4083-82fc-456717df051f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:19.503003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7be3e2c8-cb67-4083-82fc-456717df051f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:19.584545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 269/543] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:19.620928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d94b11-b748-4e7b-b942-fc4f52bfccd9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:19.677911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:19.696337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140248e-bce3-4d63-ad2f-0bc3d121fa4d None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.714931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140248e-bce3-4d63-ad2f-0bc3d121fa4d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.714931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140248e-bce3-4d63-ad2f-0bc3d121fa4d None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.714931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 270/544] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:37:19.714931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c89b6c-4655-4782-af38-9f1e07601f5e None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.714931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c89b6c-4655-4782-af38-9f1e07601f5e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.714931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c89b6c-4655-4782-af38-9f1e07601f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.722669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-74c89b6c-4655-4782-af38-9f1e07601f5e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:19.723899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-74c89b6c-4655-4782-af38-9f1e07601f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:19.748724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:19.777596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d94b11-b748-4e7b-b942-fc4f52bfccd9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:19.815094 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 271/545] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:37:19.837107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdc6f3-dd3d-4cff-9d61-9931369755a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:19.837937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc6f3-dd3d-4cff-9d61-9931369755a5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:19.838219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc6f3-dd3d-4cff-9d61-9931369755a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:19.838494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc6f3-dd3d-4cff-9d61-9931369755a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:19.847846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c89b6c-4655-4782-af38-9f1e07601f5e None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:19.854089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bdc6f3-dd3d-4cff-9d61-9931369755a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-49bdc6f3-dd3d-4cff-9d61-9931369755a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:19.868687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:19.869614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:19.870396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 275/546] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:19.878140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdc6f3-dd3d-4cff-9d61-9931369755a5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:20.007285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c89b6c-4655-4782-af38-9f1e07601f5e None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:20.047999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad95b5f-da6a-4a16-87ce-5692c6e93fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:20.048836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad95b5f-da6a-4a16-87ce-5692c6e93fda None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:20.049576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad95b5f-da6a-4a16-87ce-5692c6e93fda None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:20.049862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad95b5f-da6a-4a16-87ce-5692c6e93fda None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:20.078910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad95b5f-da6a-4a16-87ce-5692c6e93fda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-5ad95b5f-da6a-4a16-87ce-5692c6e93fda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:20.086585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:20.087454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:20.088220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 272/547] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:20.089318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdc6f3-dd3d-4cff-9d61-9931369755a5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:20.093270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:20.101008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad95b5f-da6a-4a16-87ce-5692c6e93fda None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:20.131669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:20.178951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f9f95-47f1-41ad-a44d-b3e20b94fcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:20.179629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f9f95-47f1-41ad-a44d-b3e20b94fcba None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:20.179913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f9f95-47f1-41ad-a44d-b3e20b94fcba None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:20.180197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f9f95-47f1-41ad-a44d-b3e20b94fcba None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:20.180981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f9f95-47f1-41ad-a44d-b3e20b94fcba None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:20.181291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f9f95-47f1-41ad-a44d-b3e20b94fcba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:20.182660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f9f95-47f1-41ad-a44d-b3e20b94fcba None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:20.259496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 276/548] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:20.265195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f9f95-47f1-41ad-a44d-b3e20b94fcba None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:20.359846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad95b5f-da6a-4a16-87ce-5692c6e93fda None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:20.428598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cca6c8-8b86-4968-bb25-538874c45aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:20.430349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cca6c8-8b86-4968-bb25-538874c45aa4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:20.431095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cca6c8-8b86-4968-bb25-538874c45aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:20.433084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cca6c8-8b86-4968-bb25-538874c45aa4 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:20.434815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cca6c8-8b86-4968-bb25-538874c45aa4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:20.435558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cca6c8-8b86-4968-bb25-538874c45aa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:20.438503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cca6c8-8b86-4968-bb25-538874c45aa4 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:20.455611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 273/549] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:20.462434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cca6c8-8b86-4968-bb25-538874c45aa4 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:20.476069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f9f95-47f1-41ad-a44d-b3e20b94fcba None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:20.522553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95bc8c-01dc-474e-b2ad-56d91f6c511c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:20.523833 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95bc8c-01dc-474e-b2ad-56d91f6c511c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:20.524543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95bc8c-01dc-474e-b2ad-56d91f6c511c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:20.524973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95bc8c-01dc-474e-b2ad-56d91f6c511c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:20.539995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95bc8c-01dc-474e-b2ad-56d91f6c511c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:20.541657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95bc8c-01dc-474e-b2ad-56d91f6c511c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:20.547249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-3b95bc8c-01dc-474e-b2ad-56d91f6c511c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:20.566850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:20.567908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:20.569113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:20.569113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:20.569113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:20.569113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:20.569740 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 277/550] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:20.570992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f92cb6-7321-4014-aa7d-5836fc711005 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:20.571276 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f92cb6-7321-4014-aa7d-5836fc711005 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:20.571550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f92cb6-7321-4014-aa7d-5836fc711005 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:20.572990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 278/551] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:20.577124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cca6c8-8b86-4968-bb25-538874c45aa4 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:20.577352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f92cb6-7321-4014-aa7d-5836fc711005 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:20.589320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:20.623936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee2bfc-4df1-43a3-a336-292d83392503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:20.625971 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee2bfc-4df1-43a3-a336-292d83392503 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:20.626505 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee2bfc-4df1-43a3-a336-292d83392503 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:20.628035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee2bfc-4df1-43a3-a336-292d83392503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:20.648581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee2bfc-4df1-43a3-a336-292d83392503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:20.650363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee2bfc-4df1-43a3-a336-292d83392503 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-e1ee2bfc-4df1-43a3-a336-292d83392503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:20.675520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:20.676721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:20.677594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:20.677594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:20.677594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:20.677594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:20.680139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 274/552] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:20.681605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13184f0a-aa94-4ab9-b470-4c91ae987044 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:20.682101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13184f0a-aa94-4ab9-b470-4c91ae987044 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:20.682578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13184f0a-aa94-4ab9-b470-4c91ae987044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:20.702612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-13184f0a-aa94-4ab9-b470-4c91ae987044 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:20.704134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-13184f0a-aa94-4ab9-b470-4c91ae987044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:20.710318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f92cb6-7321-4014-aa7d-5836fc711005 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:20.748586 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7770b-4770-4727-8477-d01cf6e900d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:20.750170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7770b-4770-4727-8477-d01cf6e900d3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:20.750791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7770b-4770-4727-8477-d01cf6e900d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:20.751269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7770b-4770-4727-8477-d01cf6e900d3 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:20.752220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7770b-4770-4727-8477-d01cf6e900d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:20.752869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7770b-4770-4727-8477-d01cf6e900d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:20.754549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7770b-4770-4727-8477-d01cf6e900d3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:20.769390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 279/553] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:20.770614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7770b-4770-4727-8477-d01cf6e900d3 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:20.788469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 275/554] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:20.788469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13184f0a-aa94-4ab9-b470-4c91ae987044 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:20.844947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:20.880166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:20.890703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13184f0a-aa94-4ab9-b470-4c91ae987044 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:20.927924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a80fa6-b1a3-4753-b1ec-6149b7439594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:20.928793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a80fa6-b1a3-4753-b1ec-6149b7439594 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:20.929088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a80fa6-b1a3-4753-b1ec-6149b7439594 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:20.929430 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a80fa6-b1a3-4753-b1ec-6149b7439594 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:20.930562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7770b-4770-4727-8477-d01cf6e900d3 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:20.937965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a80fa6-b1a3-4753-b1ec-6149b7439594 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:20.939824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a80fa6-b1a3-4753-b1ec-6149b7439594 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-f2a80fa6-b1a3-4753-b1ec-6149b7439594 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:20.955953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:20.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:20.957898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 276/555] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:20.958490 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a80fa6-b1a3-4753-b1ec-6149b7439594 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:20.974298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bcc27-512e-49fd-bde7-4fbae843111d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:20.975372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bcc27-512e-49fd-bde7-4fbae843111d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:20.975839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bcc27-512e-49fd-bde7-4fbae843111d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:20.976471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bcc27-512e-49fd-bde7-4fbae843111d None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:20.977381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bcc27-512e-49fd-bde7-4fbae843111d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:20.977896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bcc27-512e-49fd-bde7-4fbae843111d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:20.979670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bcc27-512e-49fd-bde7-4fbae843111d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:20.994528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 280/556] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:20.995883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bcc27-512e-49fd-bde7-4fbae843111d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:21.069407 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a80fa6-b1a3-4753-b1ec-6149b7439594 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:21.088333 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:21.108608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68522420-2222-46e4-ba27-5492d905e5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:21.109548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68522420-2222-46e4-ba27-5492d905e5a5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.109871 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68522420-2222-46e4-ba27-5492d905e5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.110195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68522420-2222-46e4-ba27-5492d905e5a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.120277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bcc27-512e-49fd-bde7-4fbae843111d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:21.121531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68522420-2222-46e4-ba27-5492d905e5a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-68522420-2222-46e4-ba27-5492d905e5a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:21.130351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:21.131475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:21.132910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:21.138257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 277/557] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:21.143565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68522420-2222-46e4-ba27-5492d905e5a5 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:21.163043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:21.172036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b32af-acbf-4954-8136-c79cebc291f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:21.173132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b32af-acbf-4954-8136-c79cebc291f5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.173625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b32af-acbf-4954-8136-c79cebc291f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.174105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b32af-acbf-4954-8136-c79cebc291f5 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.175338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b32af-acbf-4954-8136-c79cebc291f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:21.175853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b32af-acbf-4954-8136-c79cebc291f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:21.179232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b32af-acbf-4954-8136-c79cebc291f5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:21.199061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 281/558] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 23 15:37:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:21.202016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b32af-acbf-4954-8136-c79cebc291f5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:21.248086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68522420-2222-46e4-ba27-5492d905e5a5 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:21.288098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697344f9-149e-4d40-af7e-5ee3437f12a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:21.288960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697344f9-149e-4d40-af7e-5ee3437f12a3 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.289244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697344f9-149e-4d40-af7e-5ee3437f12a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.289527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697344f9-149e-4d40-af7e-5ee3437f12a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.298907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697344f9-149e-4d40-af7e-5ee3437f12a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:21.299767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697344f9-149e-4d40-af7e-5ee3437f12a3 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-697344f9-149e-4d40-af7e-5ee3437f12a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:21.316143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:21.317149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:21.318165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 278/559] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 23 15:37:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:21.318761 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697344f9-149e-4d40-af7e-5ee3437f12a3 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:21.331042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b32af-acbf-4954-8136-c79cebc291f5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:21.361133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:21.370122 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9af766-6855-45d4-a052-c5e450c60a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:21.371003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9af766-6855-45d4-a052-c5e450c60a5f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.371488 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9af766-6855-45d4-a052-c5e450c60a5f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.371912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9af766-6855-45d4-a052-c5e450c60a5f None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.372846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9af766-6855-45d4-a052-c5e450c60a5f None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:21.373338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9af766-6855-45d4-a052-c5e450c60a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:21.375439 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9af766-6855-45d4-a052-c5e450c60a5f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:21.388749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 282/560] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:21.389820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9af766-6855-45d4-a052-c5e450c60a5f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:21.427642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697344f9-149e-4d40-af7e-5ee3437f12a3 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:21.451357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:21.465077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b2bc0-c1df-4f15-9cb5-36fd9098eeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:21.465863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2bc0-c1df-4f15-9cb5-36fd9098eeab None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.466216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2bc0-c1df-4f15-9cb5-36fd9098eeab None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.466542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2bc0-c1df-4f15-9cb5-36fd9098eeab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.506320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9af766-6855-45d4-a052-c5e450c60a5f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:21.540567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd348b-9b7a-440c-b11d-8169615aeaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:21.541247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd348b-9b7a-440c-b11d-8169615aeaad None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.541534 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd348b-9b7a-440c-b11d-8169615aeaad None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.541811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd348b-9b7a-440c-b11d-8169615aeaad None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.542649 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd348b-9b7a-440c-b11d-8169615aeaad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:21.542961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd348b-9b7a-440c-b11d-8169615aeaad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:21.544397 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd348b-9b7a-440c-b11d-8169615aeaad None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:21.544503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b2bc0-c1df-4f15-9cb5-36fd9098eeab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:21.546610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b2bc0-c1df-4f15-9cb5-36fd9098eeab None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:21.568023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 283/561] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 23 15:37:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:21.568023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd348b-9b7a-440c-b11d-8169615aeaad None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:21.597564 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:21.664138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 279/562] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:21.664138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323cf59-d5d2-4830-9c57-13780ca4ef7a None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.664138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323cf59-d5d2-4830-9c57-13780ca4ef7a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.664138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323cf59-d5d2-4830-9c57-13780ca4ef7a None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.664138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 280/563] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:21.664138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323cf59-d5d2-4830-9c57-13780ca4ef7a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:21.676097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd348b-9b7a-440c-b11d-8169615aeaad None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:21.683693 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:21.712696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d3803-6eef-4d02-83cd-4483a6e66de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:21.713501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d3803-6eef-4d02-83cd-4483a6e66de9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.714028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d3803-6eef-4d02-83cd-4483a6e66de9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.714028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d3803-6eef-4d02-83cd-4483a6e66de9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.725897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d3803-6eef-4d02-83cd-4483a6e66de9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:21.727497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d3803-6eef-4d02-83cd-4483a6e66de9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-081d3803-6eef-4d02-83cd-4483a6e66de9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:21.745589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:21.746493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:21.747366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:21.747366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:21.747366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:21.747366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:21.747366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 284/564] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:21.747646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d3803-6eef-4d02-83cd-4483a6e66de9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:21.781691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323cf59-d5d2-4830-9c57-13780ca4ef7a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:21.819512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb839e-b073-4ec1-940f-dbab7f4ca909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:21.820300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb839e-b073-4ec1-940f-dbab7f4ca909 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.820625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb839e-b073-4ec1-940f-dbab7f4ca909 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.820878 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb839e-b073-4ec1-940f-dbab7f4ca909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.828676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:21.832259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb839e-b073-4ec1-940f-dbab7f4ca909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:21.833762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb839e-b073-4ec1-940f-dbab7f4ca909 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-d1fb839e-b073-4ec1-940f-dbab7f4ca909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:21.851436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:21.852325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:21.853250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:21.853250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:21.853250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:21.853250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:21.856042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d3803-6eef-4d02-83cd-4483a6e66de9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:21.857320 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 281/565] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:21.859112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e3042-1b9e-413b-820d-ecd1df6dbdaa None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.859414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e3042-1b9e-413b-820d-ecd1df6dbdaa None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.859912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e3042-1b9e-413b-820d-ecd1df6dbdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.876818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-217e3042-1b9e-413b-820d-ecd1df6dbdaa None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:21.877989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-217e3042-1b9e-413b-820d-ecd1df6dbdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:21.893215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba80dc-7ada-4b30-8560-16e97b3e448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:21.893991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba80dc-7ada-4b30-8560-16e97b3e448f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:21.894332 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba80dc-7ada-4b30-8560-16e97b3e448f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:21.894617 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba80dc-7ada-4b30-8560-16e97b3e448f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:21.904665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba80dc-7ada-4b30-8560-16e97b3e448f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:21.906657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba80dc-7ada-4b30-8560-16e97b3e448f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-5cba80dc-7ada-4b30-8560-16e97b3e448f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:21.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:21.926017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:21.926900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:21.926900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:21.926900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:21.926900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:21.926900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 285/566] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:21.927181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba80dc-7ada-4b30-8560-16e97b3e448f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:21.940143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:21.956671 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 282/567] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:21.957559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e3042-1b9e-413b-820d-ecd1df6dbdaa None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.026158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba80dc-7ada-4b30-8560-16e97b3e448f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.055264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e3042-1b9e-413b-820d-ecd1df6dbdaa None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.059863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357c8d0-2e0f-4cef-a01a-cc40f62ec1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.060547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357c8d0-2e0f-4cef-a01a-cc40f62ec1a0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.060827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357c8d0-2e0f-4cef-a01a-cc40f62ec1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.061108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357c8d0-2e0f-4cef-a01a-cc40f62ec1a0 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.061846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357c8d0-2e0f-4cef-a01a-cc40f62ec1a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:22.062150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357c8d0-2e0f-4cef-a01a-cc40f62ec1a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.063508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357c8d0-2e0f-4cef-a01a-cc40f62ec1a0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.074273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 286/568] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:22.074733 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:22.075048 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357c8d0-2e0f-4cef-a01a-cc40f62ec1a0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.093633 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc415afc-06c5-4cc2-b350-320ee27311ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.094303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc415afc-06c5-4cc2-b350-320ee27311ca None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.094587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc415afc-06c5-4cc2-b350-320ee27311ca None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.094864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc415afc-06c5-4cc2-b350-320ee27311ca None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.096870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc415afc-06c5-4cc2-b350-320ee27311ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:22.096870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc415afc-06c5-4cc2-b350-320ee27311ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.097306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc415afc-06c5-4cc2-b350-320ee27311ca None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.108439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 283/569] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:22.109569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e93006-064f-4688-8967-864e14895359 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.109860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e93006-064f-4688-8967-864e14895359 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.110142 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e93006-064f-4688-8967-864e14895359 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.111355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 284/570] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:22.112037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e93006-064f-4688-8967-864e14895359 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.147481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:22.208462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357c8d0-2e0f-4cef-a01a-cc40f62ec1a0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.241282 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd03047-75a5-4bcd-b7de-17fe7944e3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.241942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03047-75a5-4bcd-b7de-17fe7944e3d9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.242226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03047-75a5-4bcd-b7de-17fe7944e3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.242340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e93006-064f-4688-8967-864e14895359 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.242501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03047-75a5-4bcd-b7de-17fe7944e3d9 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.243225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd03047-75a5-4bcd-b7de-17fe7944e3d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:22.243537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd03047-75a5-4bcd-b7de-17fe7944e3d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.244933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd03047-75a5-4bcd-b7de-17fe7944e3d9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.256406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 287/571] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:22.257148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd03047-75a5-4bcd-b7de-17fe7944e3d9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.276576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad7c18-119d-43f9-9256-de86ad122260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.277371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad7c18-119d-43f9-9256-de86ad122260 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.277653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad7c18-119d-43f9-9256-de86ad122260 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.277928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad7c18-119d-43f9-9256-de86ad122260 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.289271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad7c18-119d-43f9-9256-de86ad122260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-6bad7c18-119d-43f9-9256-de86ad122260 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:22.296830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:22.297886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:22.298703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 285/572] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:22.300102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad7c18-119d-43f9-9256-de86ad122260 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.344173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:22.355368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd03047-75a5-4bcd-b7de-17fe7944e3d9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.383156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:22.388935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1075246-f4d1-4807-a85e-3dab719e1d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.390255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1075246-f4d1-4807-a85e-3dab719e1d98 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.390618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1075246-f4d1-4807-a85e-3dab719e1d98 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.394259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1075246-f4d1-4807-a85e-3dab719e1d98 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.397856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1075246-f4d1-4807-a85e-3dab719e1d98 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.399261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1075246-f4d1-4807-a85e-3dab719e1d98 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.401467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad7c18-119d-43f9-9256-de86ad122260 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-c1075246-f4d1-4807-a85e-3dab719e1d98 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:22.414334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:22.415468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:22.416509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 288/573] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:22.417105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efade9b1-7091-4880-b215-b95fe0820c50 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.417105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efade9b1-7091-4880-b215-b95fe0820c50 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.417372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efade9b1-7091-4880-b215-b95fe0820c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.433342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-efade9b1-7091-4880-b215-b95fe0820c50 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:22.434525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-efade9b1-7091-4880-b215-b95fe0820c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:22.441699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42ce5a-ffda-4818-9b20-ffffe29167cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.442492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42ce5a-ffda-4818-9b20-ffffe29167cf None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.442772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42ce5a-ffda-4818-9b20-ffffe29167cf None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.443046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42ce5a-ffda-4818-9b20-ffffe29167cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.449698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42ce5a-ffda-4818-9b20-ffffe29167cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.451148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42ce5a-ffda-4818-9b20-ffffe29167cf None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ea42ce5a-ffda-4818-9b20-ffffe29167cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:22.465552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:22.466428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:22.467295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:22.467295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:22.467295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:22.467295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:22.467295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:22.467295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:22.467295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:22.467295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:22.467295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 286/574] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:22.467823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42ce5a-ffda-4818-9b20-ffffe29167cf None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.502352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 289/575] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:22.503167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efade9b1-7091-4880-b215-b95fe0820c50 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.571793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:22.574690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42ce5a-ffda-4818-9b20-ffffe29167cf None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.605148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efade9b1-7091-4880-b215-b95fe0820c50 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.607921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662530e-d7de-4b40-ad28-e39512a46579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.608780 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662530e-d7de-4b40-ad28-e39512a46579 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.609061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662530e-d7de-4b40-ad28-e39512a46579 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.609333 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662530e-d7de-4b40-ad28-e39512a46579 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.615827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662530e-d7de-4b40-ad28-e39512a46579 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.617282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662530e-d7de-4b40-ad28-e39512a46579 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.625971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-9662530e-d7de-4b40-ad28-e39512a46579 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:22.630925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:22.631770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:22.632677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:22.632677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:22.632677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:22.632677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:22.632677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:22.632677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:22.632677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:22.632677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:22.632677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 287/576] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:22.633204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662530e-d7de-4b40-ad28-e39512a46579 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.644308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab98e43-de1b-4168-81f9-4509c718db64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.645011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab98e43-de1b-4168-81f9-4509c718db64 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.645297 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab98e43-de1b-4168-81f9-4509c718db64 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.645576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab98e43-de1b-4168-81f9-4509c718db64 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.646331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab98e43-de1b-4168-81f9-4509c718db64 None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:22.646643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab98e43-de1b-4168-81f9-4509c718db64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.647994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab98e43-de1b-4168-81f9-4509c718db64 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.655763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 290/577] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:22.656633 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab98e43-de1b-4168-81f9-4509c718db64 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.732576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662530e-d7de-4b40-ad28-e39512a46579 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.754376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab98e43-de1b-4168-81f9-4509c718db64 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.766021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3737ebb-2a51-48f8-bc8d-b2b0586d277b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.766689 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3737ebb-2a51-48f8-bc8d-b2b0586d277b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.766978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3737ebb-2a51-48f8-bc8d-b2b0586d277b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.767263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3737ebb-2a51-48f8-bc8d-b2b0586d277b None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.768091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3737ebb-2a51-48f8-bc8d-b2b0586d277b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:22.768431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3737ebb-2a51-48f8-bc8d-b2b0586d277b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.769862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3737ebb-2a51-48f8-bc8d-b2b0586d277b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.786390 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 288/578] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:22.787180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3737ebb-2a51-48f8-bc8d-b2b0586d277b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.787288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003340d0-10fb-4e53-b1bd-1702fb75048c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.787998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003340d0-10fb-4e53-b1bd-1702fb75048c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.788280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003340d0-10fb-4e53-b1bd-1702fb75048c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.788582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003340d0-10fb-4e53-b1bd-1702fb75048c None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.789364 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003340d0-10fb-4e53-b1bd-1702fb75048c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:22.789677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003340d0-10fb-4e53-b1bd-1702fb75048c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.791232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003340d0-10fb-4e53-b1bd-1702fb75048c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.803490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 291/579] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:22.804260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003340d0-10fb-4e53-b1bd-1702fb75048c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.809281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:22.846322 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:22.894172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3737ebb-2a51-48f8-bc8d-b2b0586d277b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.901117 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003340d0-10fb-4e53-b1bd-1702fb75048c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:22.927529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723638f-b169-4d3b-a0d4-716ef81a22ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.928324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723638f-b169-4d3b-a0d4-716ef81a22ce None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.928642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723638f-b169-4d3b-a0d4-716ef81a22ce None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.928948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723638f-b169-4d3b-a0d4-716ef81a22ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.933585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9f960-e336-4d84-bb7a-628d78af93b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:22.934237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9f960-e336-4d84-bb7a-628d78af93b4 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:22.934515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9f960-e336-4d84-bb7a-628d78af93b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:22.934787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9f960-e336-4d84-bb7a-628d78af93b4 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:22.935564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9f960-e336-4d84-bb7a-628d78af93b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:22.935873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9f960-e336-4d84-bb7a-628d78af93b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:22.937864 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9f960-e336-4d84-bb7a-628d78af93b4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:22.939400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723638f-b169-4d3b-a0d4-716ef81a22ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-7723638f-b169-4d3b-a0d4-716ef81a22ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:22.947088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:22.948194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:22.953984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 289/580] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:22.954831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 292/581] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:22.954923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723638f-b169-4d3b-a0d4-716ef81a22ce None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:22.955005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9f960-e336-4d84-bb7a-628d78af93b4 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.054670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9f960-e336-4d84-bb7a-628d78af93b4 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.056314 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:23.089703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1a2a6-9a16-40d2-8a3d-584078d8556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.090491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1a2a6-9a16-40d2-8a3d-584078d8556a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.090768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1a2a6-9a16-40d2-8a3d-584078d8556a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.091048 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1a2a6-9a16-40d2-8a3d-584078d8556a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.101480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1a2a6-9a16-40d2-8a3d-584078d8556a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.102896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1a2a6-9a16-40d2-8a3d-584078d8556a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-d9f1a2a6-9a16-40d2-8a3d-584078d8556a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:23.120641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:23.121545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:23.122415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:23.122415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:23.122415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:23.122415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:23.122415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 293/582] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:23.122699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1a2a6-9a16-40d2-8a3d-584078d8556a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.192610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:23.213007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723638f-b169-4d3b-a0d4-716ef81a22ce None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.246547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1a2a6-9a16-40d2-8a3d-584078d8556a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.247551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c744f-0e3f-41a4-a62d-27f0493d5979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.248342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c744f-0e3f-41a4-a62d-27f0493d5979 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.248659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c744f-0e3f-41a4-a62d-27f0493d5979 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.248968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c744f-0e3f-41a4-a62d-27f0493d5979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.258797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c744f-0e3f-41a4-a62d-27f0493d5979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.260527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c744f-0e3f-41a4-a62d-27f0493d5979 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-645c744f-0e3f-41a4-a62d-27f0493d5979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:23.277554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:23.278476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:23.279356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:23.279356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:23.279356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:23.279356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:23.279356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 290/583] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:23.280685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c744f-0e3f-41a4-a62d-27f0493d5979 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.281793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ccf8a-1629-44fe-a3de-2421dd12aa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.282680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ccf8a-1629-44fe-a3de-2421dd12aa5c None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.282961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ccf8a-1629-44fe-a3de-2421dd12aa5c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.283236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ccf8a-1629-44fe-a3de-2421dd12aa5c None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/b9129f0b362a4945a0eca7cc33571abd/roles/527cc946916f4976ac459aae21d85772` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.299262 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ccf8a-1629-44fe-a3de-2421dd12aa5c None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=b9129f0b362a4945a0eca7cc33571abd, role_id=527cc946916f4976ac459aae21d85772)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.300721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ccf8a-1629-44fe-a3de-2421dd12aa5c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.362714 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 294/584] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:37:23 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/b9129f0b362a4945a0eca7cc33571abd/roles/527cc946916f4976ac459aae21d85772 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:23.363326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ccf8a-1629-44fe-a3de-2421dd12aa5c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.366617 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:23.387327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c744f-0e3f-41a4-a62d-27f0493d5979 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.422090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:23.429596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93619e69-b7ba-4d43-a3df-bece3fd04f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.430477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93619e69-b7ba-4d43-a3df-bece3fd04f16 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.431199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93619e69-b7ba-4d43-a3df-bece3fd04f16 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.431488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93619e69-b7ba-4d43-a3df-bece3fd04f16 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.432598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93619e69-b7ba-4d43-a3df-bece3fd04f16 None admin] RBAC: Adding query filter params (name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:23.432914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93619e69-b7ba-4d43-a3df-bece3fd04f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.435119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93619e69-b7ba-4d43-a3df-bece3fd04f16 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.457663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 291/585] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:23.457663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93619e69-b7ba-4d43-a3df-bece3fd04f16 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.481319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ccf8a-1629-44fe-a3de-2421dd12aa5c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.519546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbb1c1-f2af-4ca6-8743-7374eeba905e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.520221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbb1c1-f2af-4ca6-8743-7374eeba905e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.520539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbb1c1-f2af-4ca6-8743-7374eeba905e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.520822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbb1c1-f2af-4ca6-8743-7374eeba905e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.521694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbb1c1-f2af-4ca6-8743-7374eeba905e None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=e57de80064b74ad59d33abb83a5d17f1) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:23.521993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbb1c1-f2af-4ca6-8743-7374eeba905e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.523469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbb1c1-f2af-4ca6-8743-7374eeba905e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.529496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 295/586] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:23 2022] GET /identity/v3/role_assignments?user.id=e57de80064b74ad59d33abb83a5d17f1&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:23.530927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbb1c1-f2af-4ca6-8743-7374eeba905e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.588869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93619e69-b7ba-4d43-a3df-bece3fd04f16 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.631912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:23.632480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbedc56-2577-4b99-9ad9-ee3bbb5319c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.632944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbedc56-2577-4b99-9ad9-ee3bbb5319c8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.633235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbedc56-2577-4b99-9ad9-ee3bbb5319c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.633525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbedc56-2577-4b99-9ad9-ee3bbb5319c8 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.634274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbedc56-2577-4b99-9ad9-ee3bbb5319c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:23.634587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbedc56-2577-4b99-9ad9-ee3bbb5319c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.636260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbedc56-2577-4b99-9ad9-ee3bbb5319c8 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.649796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 292/587] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:23.655222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbedc56-2577-4b99-9ad9-ee3bbb5319c8 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.677264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbb1c1-f2af-4ca6-8743-7374eeba905e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.710268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:23.719172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc0d5c-38d7-4c2c-a0e1-7b18d06a648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.720299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc0d5c-38d7-4c2c-a0e1-7b18d06a648a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.726665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc0d5c-38d7-4c2c-a0e1-7b18d06a648a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.726665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc0d5c-38d7-4c2c-a0e1-7b18d06a648a None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.726665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc0d5c-38d7-4c2c-a0e1-7b18d06a648a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:23.726665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc0d5c-38d7-4c2c-a0e1-7b18d06a648a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.726665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc0d5c-38d7-4c2c-a0e1-7b18d06a648a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.739140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 296/588] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:23.740315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc0d5c-38d7-4c2c-a0e1-7b18d06a648a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.765412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbedc56-2577-4b99-9ad9-ee3bbb5319c8 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.818287 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0a5ab-5b65-4c00-adfa-572a9c3ace35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.819093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0a5ab-5b65-4c00-adfa-572a9c3ace35 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.819376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0a5ab-5b65-4c00-adfa-572a9c3ace35 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.819655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0a5ab-5b65-4c00-adfa-572a9c3ace35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.833703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0a5ab-5b65-4c00-adfa-572a9c3ace35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.835161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0a5ab-5b65-4c00-adfa-572a9c3ace35 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.852078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc0d5c-38d7-4c2c-a0e1-7b18d06a648a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-a1d0a5ab-5b65-4c00-adfa-572a9c3ace35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:23.854705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:23.855568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:23.856499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:23.856499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:23.856499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:23.856499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:23.856499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 293/589] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:23.857181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0a5ab-5b65-4c00-adfa-572a9c3ace35 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.887499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:23.899205 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba2826-92a9-4c89-8ee5-76235dc64e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:23.900074 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba2826-92a9-4c89-8ee5-76235dc64e13 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:23.900393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba2826-92a9-4c89-8ee5-76235dc64e13 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:23.900793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba2826-92a9-4c89-8ee5-76235dc64e13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:23.908877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba2826-92a9-4c89-8ee5-76235dc64e13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:23.910437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba2826-92a9-4c89-8ee5-76235dc64e13 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-09ba2826-92a9-4c89-8ee5-76235dc64e13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:23.927255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:23.928259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:23.929796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:23.929796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:23.929796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:23.929796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:23.929796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:23.929796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:23.929796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:23.929796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:23.929796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 297/590] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:23.929796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba2826-92a9-4c89-8ee5-76235dc64e13 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:23.977017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0a5ab-5b65-4c00-adfa-572a9c3ace35 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:23.980586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.012824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d533d9-fbac-46bb-9230-c9b52da1a303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.013960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d533d9-fbac-46bb-9230-c9b52da1a303 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.014442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d533d9-fbac-46bb-9230-c9b52da1a303 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.015025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d533d9-fbac-46bb-9230-c9b52da1a303 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.022511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d533d9-fbac-46bb-9230-c9b52da1a303 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.024208 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d533d9-fbac-46bb-9230-c9b52da1a303 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-25d533d9-fbac-46bb-9230-c9b52da1a303 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:24.040108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:24.041088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:24.041990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:24.041990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:24.041990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:24.041990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:24.041990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:24.041990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:24.041990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:24.041990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:24.041990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 15:37:24.041990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:24.044690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 294/591] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:24.045753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d533d9-fbac-46bb-9230-c9b52da1a303 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.065350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba2826-92a9-4c89-8ee5-76235dc64e13 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.112010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e86b1-206f-4d01-bcd2-fe003aacde14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.113031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e86b1-206f-4d01-bcd2-fe003aacde14 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.113476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e86b1-206f-4d01-bcd2-fe003aacde14 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.117699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e86b1-206f-4d01-bcd2-fe003aacde14 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.117699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e86b1-206f-4d01-bcd2-fe003aacde14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:24.117699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e86b1-206f-4d01-bcd2-fe003aacde14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.117699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e86b1-206f-4d01-bcd2-fe003aacde14 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.133386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 298/592] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:24.134624 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e86b1-206f-4d01-bcd2-fe003aacde14 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.161641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d533d9-fbac-46bb-9230-c9b52da1a303 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.189579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.214193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810dbc4-caf7-4c2f-a8a7-f1c942330953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.214867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810dbc4-caf7-4c2f-a8a7-f1c942330953 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.215149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810dbc4-caf7-4c2f-a8a7-f1c942330953 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.215423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810dbc4-caf7-4c2f-a8a7-f1c942330953 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.221013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810dbc4-caf7-4c2f-a8a7-f1c942330953 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:24.221329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810dbc4-caf7-4c2f-a8a7-f1c942330953 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.223308 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810dbc4-caf7-4c2f-a8a7-f1c942330953 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.240714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 295/593] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:24.241485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810dbc4-caf7-4c2f-a8a7-f1c942330953 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.255538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e86b1-206f-4d01-bcd2-fe003aacde14 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.273772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.298394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b780c1-a7c7-4fd7-9705-0e82f7818a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.299355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b780c1-a7c7-4fd7-9705-0e82f7818a66 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.299843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b780c1-a7c7-4fd7-9705-0e82f7818a66 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.300389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b780c1-a7c7-4fd7-9705-0e82f7818a66 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.301433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b780c1-a7c7-4fd7-9705-0e82f7818a66 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:24.301943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b780c1-a7c7-4fd7-9705-0e82f7818a66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.303627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b780c1-a7c7-4fd7-9705-0e82f7818a66 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.322148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 299/594] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:24.323374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b780c1-a7c7-4fd7-9705-0e82f7818a66 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.363491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810dbc4-caf7-4c2f-a8a7-f1c942330953 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.398260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea63376-f50c-4515-a5a3-8c051e5f4a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.399416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea63376-f50c-4515-a5a3-8c051e5f4a6e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.399911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea63376-f50c-4515-a5a3-8c051e5f4a6e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.400406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea63376-f50c-4515-a5a3-8c051e5f4a6e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.408929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea63376-f50c-4515-a5a3-8c051e5f4a6e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.410743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea63376-f50c-4515-a5a3-8c051e5f4a6e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-bea63376-f50c-4515-a5a3-8c051e5f4a6e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:24.426104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:24.427306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:24.428277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:24.428277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:24.428277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:24.428277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:24.428277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:24.428277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:24.428277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:24.428277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:24.428277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:37:24.428277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:24.430704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 296/595] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:24.436128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea63376-f50c-4515-a5a3-8c051e5f4a6e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.439317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b780c1-a7c7-4fd7-9705-0e82f7818a66 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.482755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c1ad5-7a62-4917-9718-4d673869334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.484246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1ad5-7a62-4917-9718-4d673869334a None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.484578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1ad5-7a62-4917-9718-4d673869334a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.484864 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1ad5-7a62-4917-9718-4d673869334a None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/0d30c8d1f883400ba397ea09d42a0aa8/roles/527cc946916f4976ac459aae21d85772` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.505703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c1ad5-7a62-4917-9718-4d673869334a None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=0d30c8d1f883400ba397ea09d42a0aa8, role_id=527cc946916f4976ac459aae21d85772)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.507229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c1ad5-7a62-4917-9718-4d673869334a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.537601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.549473 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea63376-f50c-4515-a5a3-8c051e5f4a6e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.580567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.586240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bca1ec-e9ed-4712-9cdb-d5dcf7b8d568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.587044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bca1ec-e9ed-4712-9cdb-d5dcf7b8d568 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.587328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bca1ec-e9ed-4712-9cdb-d5dcf7b8d568 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.587603 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bca1ec-e9ed-4712-9cdb-d5dcf7b8d568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.596636 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 300/596] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:37:24 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/0d30c8d1f883400ba397ea09d42a0aa8/roles/527cc946916f4976ac459aae21d85772 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:24.598039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c1ad5-7a62-4917-9718-4d673869334a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.614859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bca1ec-e9ed-4712-9cdb-d5dcf7b8d568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.616784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bca1ec-e9ed-4712-9cdb-d5dcf7b8d568 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-a2bca1ec-e9ed-4712-9cdb-d5dcf7b8d568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:24.645403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:24.646370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:24.647432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:24.647432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:24.647432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:24.647432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:24.647432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 297/597] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:24.652226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bca1ec-e9ed-4712-9cdb-d5dcf7b8d568 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.830329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bca1ec-e9ed-4712-9cdb-d5dcf7b8d568 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.852591 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c1ad5-7a62-4917-9718-4d673869334a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:24.875469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fb79f-bd89-455d-a19c-c5b0fa309155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.876143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fb79f-bd89-455d-a19c-c5b0fa309155 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.876449 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fb79f-bd89-455d-a19c-c5b0fa309155 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.876726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fb79f-bd89-455d-a19c-c5b0fa309155 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.877451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fb79f-bd89-455d-a19c-c5b0fa309155 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:24.877755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fb79f-bd89-455d-a19c-c5b0fa309155 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.879128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fb79f-bd89-455d-a19c-c5b0fa309155 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.881696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:24.896399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 298/598] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:24.899511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fb79f-bd89-455d-a19c-c5b0fa309155 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.904853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d895610-74d5-4e48-b6de-1d10e40c66bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:24.905556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d895610-74d5-4e48-b6de-1d10e40c66bc None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:24.905841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d895610-74d5-4e48-b6de-1d10e40c66bc None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:24.906120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d895610-74d5-4e48-b6de-1d10e40c66bc None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:24.906970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d895610-74d5-4e48-b6de-1d10e40c66bc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:24.907279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d895610-74d5-4e48-b6de-1d10e40c66bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:24.908966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d895610-74d5-4e48-b6de-1d10e40c66bc None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:24.930901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 301/599] 10.222.0.23 () {56 vars in 1139 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:24.932598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d895610-74d5-4e48-b6de-1d10e40c66bc None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:24.950111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:25.059467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d895610-74d5-4e48-b6de-1d10e40c66bc None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.093485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fb79f-bd89-455d-a19c-c5b0fa309155 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.114259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a952cbe-7fdb-49e1-a286-985f691d98aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.115072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a952cbe-7fdb-49e1-a286-985f691d98aa None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.115355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a952cbe-7fdb-49e1-a286-985f691d98aa None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.115632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a952cbe-7fdb-49e1-a286-985f691d98aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.137512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a952cbe-7fdb-49e1-a286-985f691d98aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.138953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a952cbe-7fdb-49e1-a286-985f691d98aa None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.152245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ac2a3-8fd1-4987-9309-743bd22b85de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.153073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ac2a3-8fd1-4987-9309-743bd22b85de None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.153353 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ac2a3-8fd1-4987-9309-743bd22b85de None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.153624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ac2a3-8fd1-4987-9309-743bd22b85de None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.160696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ac2a3-8fd1-4987-9309-743bd22b85de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.162134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ac2a3-8fd1-4987-9309-743bd22b85de None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-2a952cbe-7fdb-49e1-a286-985f691d98aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:25.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:25.165351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:25.166208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:25.166208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:25.166208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:25.166208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:25.174321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 302/600] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-184ac2a3-8fd1-4987-9309-743bd22b85de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:25.180916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:25.181774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:25.182661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:25.182661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:25.182661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:25.182661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:25.182661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:25.182661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:25.182661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:25.182661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:25.187153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 299/601] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:25.188416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a952cbe-7fdb-49e1-a286-985f691d98aa None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.188732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ac2a3-8fd1-4987-9309-743bd22b85de None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.284407 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:25.299658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:25.347346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ac2a3-8fd1-4987-9309-743bd22b85de None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.366540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a952cbe-7fdb-49e1-a286-985f691d98aa None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.385967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8785-9802-4092-b597-ad1e7a5b6acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.386638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8785-9802-4092-b597-ad1e7a5b6acf None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.386919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8785-9802-4092-b597-ad1e7a5b6acf None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.387198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8785-9802-4092-b597-ad1e7a5b6acf None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.387926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b8785-9802-4092-b597-ad1e7a5b6acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:25.388232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b8785-9802-4092-b597-ad1e7a5b6acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.390153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b8785-9802-4092-b597-ad1e7a5b6acf None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.402834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 300/602] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:25.408517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8785-9802-4092-b597-ad1e7a5b6acf None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.436319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba067a-fb7e-4922-8ecc-91c627143645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.437539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba067a-fb7e-4922-8ecc-91c627143645 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.437819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba067a-fb7e-4922-8ecc-91c627143645 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.438089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba067a-fb7e-4922-8ecc-91c627143645 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.438886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba067a-fb7e-4922-8ecc-91c627143645 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:25.439191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba067a-fb7e-4922-8ecc-91c627143645 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.440630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba067a-fb7e-4922-8ecc-91c627143645 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.467613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 303/603] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:25.473076 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba067a-fb7e-4922-8ecc-91c627143645 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.548225 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8785-9802-4092-b597-ad1e7a5b6acf None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.567854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:25.598087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23f113-d477-4bb0-a14a-acef1d804854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.599238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23f113-d477-4bb0-a14a-acef1d804854 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.599753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23f113-d477-4bb0-a14a-acef1d804854 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.600253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23f113-d477-4bb0-a14a-acef1d804854 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.606197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba067a-fb7e-4922-8ecc-91c627143645 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.617497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23f113-d477-4bb0-a14a-acef1d804854 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.619252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23f113-d477-4bb0-a14a-acef1d804854 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.648745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ed23f113-d477-4bb0-a14a-acef1d804854 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:25.651106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:25.651958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:25.652877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:25.652877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:25.652877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:25.652877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:25.652877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:25.652877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:25.652877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:25.652877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:25.652877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 301/604] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:25.658255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23f113-d477-4bb0-a14a-acef1d804854 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.666711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c65384-b31e-479d-9088-df4e000d27f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.667649 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c65384-b31e-479d-9088-df4e000d27f1 None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.667934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c65384-b31e-479d-9088-df4e000d27f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.668779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c65384-b31e-479d-9088-df4e000d27f1 None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/142e6e44d6544f6a891ed3f913de74ae/roles/527cc946916f4976ac459aae21d85772` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.699621 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c65384-b31e-479d-9088-df4e000d27f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=142e6e44d6544f6a891ed3f913de74ae, role_id=527cc946916f4976ac459aae21d85772)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.701125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c65384-b31e-479d-9088-df4e000d27f1 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.787298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23f113-d477-4bb0-a14a-acef1d804854 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:25.803149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 304/605] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:37:25 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/142e6e44d6544f6a891ed3f913de74ae/roles/527cc946916f4976ac459aae21d85772 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:25.804615 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5f50b-b246-4344-8b53-f70ca8254e6b None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.804909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5f50b-b246-4344-8b53-f70ca8254e6b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.805188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5f50b-b246-4344-8b53-f70ca8254e6b None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.810907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 305/606] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:25.813491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec282-8f13-4510-b5c4-c1968993acd3 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.819808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec282-8f13-4510-b5c4-c1968993acd3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.819808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec282-8f13-4510-b5c4-c1968993acd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.834955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-664ec282-8f13-4510-b5c4-c1968993acd3 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:25.836717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-664ec282-8f13-4510-b5c4-c1968993acd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:25.842606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fed27-20da-431d-b77b-2b9ec1b44319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:25.844615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fed27-20da-431d-b77b-2b9ec1b44319 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:25.845146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fed27-20da-431d-b77b-2b9ec1b44319 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:25.845619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fed27-20da-431d-b77b-2b9ec1b44319 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:25.846651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fed27-20da-431d-b77b-2b9ec1b44319 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:25.847287 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fed27-20da-431d-b77b-2b9ec1b44319 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:25.849325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fed27-20da-431d-b77b-2b9ec1b44319 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:25.871149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 302/607] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:25.877167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fed27-20da-431d-b77b-2b9ec1b44319 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:25.941876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:25.960858 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 306/608] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 15:37:25.996753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ec282-8f13-4510-b5c4-c1968993acd3 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.016721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fed27-20da-431d-b77b-2b9ec1b44319 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.060977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581dd2b-7303-41f4-82d2-2f99a07a9772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.062200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581dd2b-7303-41f4-82d2-2f99a07a9772 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.063457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581dd2b-7303-41f4-82d2-2f99a07a9772 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.063753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581dd2b-7303-41f4-82d2-2f99a07a9772 None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/a39f5189d63d467dbd914f553c261c01/roles/527cc946916f4976ac459aae21d85772` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.082140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581dd2b-7303-41f4-82d2-2f99a07a9772 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=a39f5189d63d467dbd914f553c261c01, role_id=527cc946916f4976ac459aae21d85772)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:26.083599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581dd2b-7303-41f4-82d2-2f99a07a9772 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:26.140567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ec282-8f13-4510-b5c4-c1968993acd3 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.148748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:26.151451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 303/609] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:37:25 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/a39f5189d63d467dbd914f553c261c01/roles/527cc946916f4976ac459aae21d85772 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:26.151451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1fcea-f049-416f-9cf3-7a5ae32eb0dd None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.151451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1fcea-f049-416f-9cf3-7a5ae32eb0dd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.152085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1fcea-f049-416f-9cf3-7a5ae32eb0dd None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.154690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 304/610] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:26.162230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e3064-31db-417c-986b-08582703afeb None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.162874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e3064-31db-417c-986b-08582703afeb None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.163476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e3064-31db-417c-986b-08582703afeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.182627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669a465-b6c9-4569-9d7e-718050df9203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.183547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669a465-b6c9-4569-9d7e-718050df9203 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.186219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669a465-b6c9-4569-9d7e-718050df9203 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.186219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669a465-b6c9-4569-9d7e-718050df9203 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.186421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-3f0e3064-31db-417c-986b-08582703afeb None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:26.187921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3f0e3064-31db-417c-986b-08582703afeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:26.198104 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669a465-b6c9-4569-9d7e-718050df9203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-f669a465-b6c9-4569-9d7e-718050df9203 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:26.208958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:26.209989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:26.210824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:26.213852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 307/611] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:26.220004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669a465-b6c9-4569-9d7e-718050df9203 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.274753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 305/612] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:26.297702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e3064-31db-417c-986b-08582703afeb None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.317089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:26.360451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669a465-b6c9-4569-9d7e-718050df9203 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.412218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a5641-5dc0-42ee-b945-5ec5169ce6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.414362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a5641-5dc0-42ee-b945-5ec5169ce6bd None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.414653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a5641-5dc0-42ee-b945-5ec5169ce6bd None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.414931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a5641-5dc0-42ee-b945-5ec5169ce6bd None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.415721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a5641-5dc0-42ee-b945-5ec5169ce6bd None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:26.416031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a5641-5dc0-42ee-b945-5ec5169ce6bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:26.417456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a5641-5dc0-42ee-b945-5ec5169ce6bd None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:26.421193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e3064-31db-417c-986b-08582703afeb None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.441570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 308/613] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:26.447901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a5641-5dc0-42ee-b945-5ec5169ce6bd None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.490399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7424de-87b3-4714-9d7d-a0da591387e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.491566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7424de-87b3-4714-9d7d-a0da591387e9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.492233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7424de-87b3-4714-9d7d-a0da591387e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.493297 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:26.493521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7424de-87b3-4714-9d7d-a0da591387e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.510745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7424de-87b3-4714-9d7d-a0da591387e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-6f7424de-87b3-4714-9d7d-a0da591387e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:26.521854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:26.522865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:26.523682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:26.526913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 306/614] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:26.533455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7424de-87b3-4714-9d7d-a0da591387e9 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.591283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a5641-5dc0-42ee-b945-5ec5169ce6bd None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.637015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:26.660556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7424de-87b3-4714-9d7d-a0da591387e9 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.678538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3a691-0f2e-41db-bde6-bbe334618333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.679272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3a691-0f2e-41db-bde6-bbe334618333 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.679549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3a691-0f2e-41db-bde6-bbe334618333 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.679815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3a691-0f2e-41db-bde6-bbe334618333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.703068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31418e-2d8f-4f80-867f-ea8037a27841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.703745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31418e-2d8f-4f80-867f-ea8037a27841 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.704080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31418e-2d8f-4f80-867f-ea8037a27841 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.704375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31418e-2d8f-4f80-867f-ea8037a27841 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.705139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31418e-2d8f-4f80-867f-ea8037a27841 None admin] RBAC: Adding query filter params (name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:26.705497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31418e-2d8f-4f80-867f-ea8037a27841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:26.706892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31418e-2d8f-4f80-867f-ea8037a27841 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:26.709448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3a691-0f2e-41db-bde6-bbe334618333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:26.710895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3a691-0f2e-41db-bde6-bbe334618333 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:26.716096 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 307/615] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:26.721751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31418e-2d8f-4f80-867f-ea8037a27841 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-27c3a691-0f2e-41db-bde6-bbe334618333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:26.749761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:26.750703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:26.751592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:26.751592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:26.751592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:26.751592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:26.751592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 309/616] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:26.756244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3a691-0f2e-41db-bde6-bbe334618333 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.825928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31418e-2d8f-4f80-867f-ea8037a27841 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:26.861401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f747fe-adad-496b-81d3-e47a4e4768f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:26.862198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f747fe-adad-496b-81d3-e47a4e4768f3 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:26.862485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f747fe-adad-496b-81d3-e47a4e4768f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:26.862760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f747fe-adad-496b-81d3-e47a4e4768f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:26.869862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:26.875625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f747fe-adad-496b-81d3-e47a4e4768f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:26.877316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f747fe-adad-496b-81d3-e47a4e4768f3 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-f1f747fe-adad-496b-81d3-e47a4e4768f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:26.895988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:26.897090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:26.898112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:26.898112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:26.898112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:26.898112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:26.898112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 308/617] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:26.903052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f747fe-adad-496b-81d3-e47a4e4768f3 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:26.905797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:26.945050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3a691-0f2e-41db-bde6-bbe334618333 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.009158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b33e7a-5c45-4fc3-9fa3-60245e0f55b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.010953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b33e7a-5c45-4fc3-9fa3-60245e0f55b4 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.011451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b33e7a-5c45-4fc3-9fa3-60245e0f55b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.011932 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b33e7a-5c45-4fc3-9fa3-60245e0f55b4 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.012901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b33e7a-5c45-4fc3-9fa3-60245e0f55b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:27.013430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b33e7a-5c45-4fc3-9fa3-60245e0f55b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.014571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f747fe-adad-496b-81d3-e47a4e4768f3 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.015335 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b33e7a-5c45-4fc3-9fa3-60245e0f55b4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.042726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 310/618] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:27.048313 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b33e7a-5c45-4fc3-9fa3-60245e0f55b4 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.056309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887db965-b2f7-4c04-bb7c-43c09c6137ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.058567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887db965-b2f7-4c04-bb7c-43c09c6137ee None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.058567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887db965-b2f7-4c04-bb7c-43c09c6137ee None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.058567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887db965-b2f7-4c04-bb7c-43c09c6137ee None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.059160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887db965-b2f7-4c04-bb7c-43c09c6137ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:27.059475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887db965-b2f7-4c04-bb7c-43c09c6137ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.060925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887db965-b2f7-4c04-bb7c-43c09c6137ee None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.082478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 309/619] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:27.088085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887db965-b2f7-4c04-bb7c-43c09c6137ee None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.143179 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.192319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887db965-b2f7-4c04-bb7c-43c09c6137ee None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.230139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4eb893-e601-4c64-9b4b-52577c70e4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.230951 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4eb893-e601-4c64-9b4b-52577c70e4f8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.231305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4eb893-e601-4c64-9b4b-52577c70e4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.231649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4eb893-e601-4c64-9b4b-52577c70e4f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.240705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4eb893-e601-4c64-9b4b-52577c70e4f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.242452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4eb893-e601-4c64-9b4b-52577c70e4f8 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-8d4eb893-e601-4c64-9b4b-52577c70e4f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:27.260168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:27.261429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:27.262357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:27.262357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:27.262357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:27.262357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:27.262357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:27.262357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:27.262357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:27.262357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:27.262357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:27.262357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:27.262891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 310/620] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:27.265898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce693db-7c6b-4582-a0fa-ffd18b077e37 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.265898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce693db-7c6b-4582-a0fa-ffd18b077e37 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.266243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce693db-7c6b-4582-a0fa-ffd18b077e37 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.271915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 311/621] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:27.272704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce693db-7c6b-4582-a0fa-ffd18b077e37 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.274864 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b33e7a-5c45-4fc3-9fa3-60245e0f55b4 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.315975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff82ff9-fcc8-43b5-a4e4-9ec0064f0438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.317263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff82ff9-fcc8-43b5-a4e4-9ec0064f0438 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.317852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff82ff9-fcc8-43b5-a4e4-9ec0064f0438 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.318324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff82ff9-fcc8-43b5-a4e4-9ec0064f0438 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.328276 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff82ff9-fcc8-43b5-a4e4-9ec0064f0438 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.330551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff82ff9-fcc8-43b5-a4e4-9ec0064f0438 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.338490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-dff82ff9-fcc8-43b5-a4e4-9ec0064f0438 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:27.351056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:27.352211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:27.353381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:27.353381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:27.353381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:27.353381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:27.353381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:27.353381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:27.353381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:27.353381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:27.353381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:27.353381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:27.359167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 311/622] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:27.359167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c12dd7-9e04-42a3-8d6d-46c08f3e6506 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.359167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c12dd7-9e04-42a3-8d6d-46c08f3e6506 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.359167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c12dd7-9e04-42a3-8d6d-46c08f3e6506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.374980 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-64c12dd7-9e04-42a3-8d6d-46c08f3e6506 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:27.376372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-64c12dd7-9e04-42a3-8d6d-46c08f3e6506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:27.408080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce693db-7c6b-4582-a0fa-ffd18b077e37 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.419765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.445238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56672a6a-4bd1-44c8-a842-a995b1479597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.446204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56672a6a-4bd1-44c8-a842-a995b1479597 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.446663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56672a6a-4bd1-44c8-a842-a995b1479597 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.447111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56672a6a-4bd1-44c8-a842-a995b1479597 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.448133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56672a6a-4bd1-44c8-a842-a995b1479597 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:27.449147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 312/623] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:27.450213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c12dd7-9e04-42a3-8d6d-46c08f3e6506 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.450602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56672a6a-4bd1-44c8-a842-a995b1479597 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.453065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56672a6a-4bd1-44c8-a842-a995b1479597 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.474086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 312/624] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:27.479833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56672a6a-4bd1-44c8-a842-a995b1479597 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.555909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c12dd7-9e04-42a3-8d6d-46c08f3e6506 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.585357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.590753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e687547-fae9-4970-b966-1e4d69dd7096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.591357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56672a6a-4bd1-44c8-a842-a995b1479597 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.591471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e687547-fae9-4970-b966-1e4d69dd7096 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.591710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e687547-fae9-4970-b966-1e4d69dd7096 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.591989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e687547-fae9-4970-b966-1e4d69dd7096 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.593427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e687547-fae9-4970-b966-1e4d69dd7096 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:27.593810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e687547-fae9-4970-b966-1e4d69dd7096 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.595233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e687547-fae9-4970-b966-1e4d69dd7096 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.612698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 313/625] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:27.615199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e687547-fae9-4970-b966-1e4d69dd7096 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.630061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d05065-4bd2-48a5-b4e3-a6ed904747c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.631238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d05065-4bd2-48a5-b4e3-a6ed904747c9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.631734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d05065-4bd2-48a5-b4e3-a6ed904747c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.632233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d05065-4bd2-48a5-b4e3-a6ed904747c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.645567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d05065-4bd2-48a5-b4e3-a6ed904747c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.647545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d05065-4bd2-48a5-b4e3-a6ed904747c9 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-04d05065-4bd2-48a5-b4e3-a6ed904747c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:27.667023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:27.667953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:27.668879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:27.668879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:27.668879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:27.668879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:27.671139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 313/626] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:27.672219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d05065-4bd2-48a5-b4e3-a6ed904747c9 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.701759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.733587 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e687547-fae9-4970-b966-1e4d69dd7096 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.771918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13250950-9bf1-4952-aead-d57789ad7e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.772843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13250950-9bf1-4952-aead-d57789ad7e55 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.773029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13250950-9bf1-4952-aead-d57789ad7e55 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.773291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13250950-9bf1-4952-aead-d57789ad7e55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.784350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d05065-4bd2-48a5-b4e3-a6ed904747c9 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.784617 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13250950-9bf1-4952-aead-d57789ad7e55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-13250950-9bf1-4952-aead-d57789ad7e55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:27.792811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:27.793762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 314/627] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:27.795002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13250950-9bf1-4952-aead-d57789ad7e55 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:27.824041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7944a-1fbc-4d3b-9562-c2bbf04db243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.825042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7944a-1fbc-4d3b-9562-c2bbf04db243 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.825529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7944a-1fbc-4d3b-9562-c2bbf04db243 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.826036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7944a-1fbc-4d3b-9562-c2bbf04db243 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.850052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.895775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7944a-1fbc-4d3b-9562-c2bbf04db243 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.898014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7944a-1fbc-4d3b-9562-c2bbf04db243 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.906416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13250950-9bf1-4952-aead-d57789ad7e55 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:27.919230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:27.945577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab1164-bd1f-4db1-ad23-ae02b11094a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:27.946478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab1164-bd1f-4db1-ad23-ae02b11094a3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:27.946919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab1164-bd1f-4db1-ad23-ae02b11094a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:27.947341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab1164-bd1f-4db1-ad23-ae02b11094a3 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:27.948213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab1164-bd1f-4db1-ad23-ae02b11094a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:27.948710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab1164-bd1f-4db1-ad23-ae02b11094a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:27.950327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab1164-bd1f-4db1-ad23-ae02b11094a3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:27.965171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 315/628] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:27.966342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab1164-bd1f-4db1-ad23-ae02b11094a3 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.001089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 314/629] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:28.002134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7944a-1fbc-4d3b-9562-c2bbf04db243 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.077185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab1164-bd1f-4db1-ad23-ae02b11094a3 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.088404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.102886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7944a-1fbc-4d3b-9562-c2bbf04db243 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.116689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c13272-f0d5-4847-a09e-f0f083e13d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.117745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c13272-f0d5-4847-a09e-f0f083e13d5f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.118218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c13272-f0d5-4847-a09e-f0f083e13d5f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.118726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c13272-f0d5-4847-a09e-f0f083e13d5f None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.119805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c13272-f0d5-4847-a09e-f0f083e13d5f None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:28.120432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c13272-f0d5-4847-a09e-f0f083e13d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.123088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c13272-f0d5-4847-a09e-f0f083e13d5f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.124838 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.134151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 316/630] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:28.135335 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c13272-f0d5-4847-a09e-f0f083e13d5f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.139937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b36af4-a6fc-477d-ba0e-38836cee92b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.141058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36af4-a6fc-477d-ba0e-38836cee92b0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.141058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36af4-a6fc-477d-ba0e-38836cee92b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.141375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36af4-a6fc-477d-ba0e-38836cee92b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.148664 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b36af4-a6fc-477d-ba0e-38836cee92b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.150173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b36af4-a6fc-477d-ba0e-38836cee92b0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-55b36af4-a6fc-477d-ba0e-38836cee92b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:28.164549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:28.165499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:28.166392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:28.166392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:28.166392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:28.166392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:28.166392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:28.166392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:28.166392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:28.166392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:28.166392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 315/631] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:28.167493 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b36af4-a6fc-477d-ba0e-38836cee92b0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.257852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c13272-f0d5-4847-a09e-f0f083e13d5f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.269550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b36af4-a6fc-477d-ba0e-38836cee92b0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.296609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285c21d-870a-42ca-ae0b-938ff2f10d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.297755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285c21d-870a-42ca-ae0b-938ff2f10d69 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.298271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285c21d-870a-42ca-ae0b-938ff2f10d69 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.298837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285c21d-870a-42ca-ae0b-938ff2f10d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.304610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc150170-961d-466a-bed2-7df6a067b413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.305529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc150170-961d-466a-bed2-7df6a067b413 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.305911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc150170-961d-466a-bed2-7df6a067b413 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.306283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc150170-961d-466a-bed2-7df6a067b413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.313609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285c21d-870a-42ca-ae0b-938ff2f10d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.315418 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285c21d-870a-42ca-ae0b-938ff2f10d69 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.317436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc150170-961d-466a-bed2-7df6a067b413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.318794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc150170-961d-466a-bed2-7df6a067b413 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-1285c21d-870a-42ca-ae0b-938ff2f10d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:28.335211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:28.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:28.337374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:28.337374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:28.337374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:28.337374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-dc150170-961d-466a-bed2-7df6a067b413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:28.337594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:28.338386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:28.339260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:28.339260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:28.339260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:28.339260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:28.339260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 316/632] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:28.339260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc150170-961d-466a-bed2-7df6a067b413 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.346592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 317/633] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:28.347787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285c21d-870a-42ca-ae0b-938ff2f10d69 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.375461 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.380485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.449399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc150170-961d-466a-bed2-7df6a067b413 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.470623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285c21d-870a-42ca-ae0b-938ff2f10d69 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.492102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03cdc2-4c69-4831-984d-8182df46b31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.493120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03cdc2-4c69-4831-984d-8182df46b31a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.493633 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03cdc2-4c69-4831-984d-8182df46b31a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.494532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03cdc2-4c69-4831-984d-8182df46b31a None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.495826 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03cdc2-4c69-4831-984d-8182df46b31a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:28.496340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03cdc2-4c69-4831-984d-8182df46b31a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.498747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03cdc2-4c69-4831-984d-8182df46b31a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.509417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3b6e3-1b68-4eee-b098-7f63604b331e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.510082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3b6e3-1b68-4eee-b098-7f63604b331e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.510362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3b6e3-1b68-4eee-b098-7f63604b331e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.510895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3b6e3-1b68-4eee-b098-7f63604b331e None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.512042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3b6e3-1b68-4eee-b098-7f63604b331e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:28.512389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3b6e3-1b68-4eee-b098-7f63604b331e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.514038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3b6e3-1b68-4eee-b098-7f63604b331e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.516690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 317/634] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:28.518228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03cdc2-4c69-4831-984d-8182df46b31a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.528097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 318/635] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:28.529461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3b6e3-1b68-4eee-b098-7f63604b331e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.623385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.626008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.632232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03cdc2-4c69-4831-984d-8182df46b31a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.645140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3b6e3-1b68-4eee-b098-7f63604b331e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.668571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3cc0-18d8-4714-899b-0b28fb83c201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.669247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3cc0-18d8-4714-899b-0b28fb83c201 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.669530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3cc0-18d8-4714-899b-0b28fb83c201 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.669807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3cc0-18d8-4714-899b-0b28fb83c201 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.670522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a3cc0-18d8-4714-899b-0b28fb83c201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:28.670833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a3cc0-18d8-4714-899b-0b28fb83c201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.672572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a3cc0-18d8-4714-899b-0b28fb83c201 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.681980 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcffad-17f8-4de7-a5bc-2af237c3b1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.683105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcffad-17f8-4de7-a5bc-2af237c3b1c5 None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.683580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcffad-17f8-4de7-a5bc-2af237c3b1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.684118 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 318/636] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:28.684503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcffad-17f8-4de7-a5bc-2af237c3b1c5 None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/e57de80064b74ad59d33abb83a5d17f1/roles/527cc946916f4976ac459aae21d85772` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.684949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3cc0-18d8-4714-899b-0b28fb83c201 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.704655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcffad-17f8-4de7-a5bc-2af237c3b1c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=e57de80064b74ad59d33abb83a5d17f1, role_id=527cc946916f4976ac459aae21d85772)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.706438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcffad-17f8-4de7-a5bc-2af237c3b1c5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.786181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 319/637] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:37:28 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/e57de80064b74ad59d33abb83a5d17f1/roles/527cc946916f4976ac459aae21d85772 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:28.787538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150d807-c418-4459-9d10-69e9b79a7f23 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.787841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150d807-c418-4459-9d10-69e9b79a7f23 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.788161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150d807-c418-4459-9d10-69e9b79a7f23 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.788515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3cc0-18d8-4714-899b-0b28fb83c201 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.789645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 320/638] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:28.790216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150d807-c418-4459-9d10-69e9b79a7f23 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:28.830193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f623bd-b547-4ac5-a783-903135a046e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.831010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f623bd-b547-4ac5-a783-903135a046e1 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.831294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f623bd-b547-4ac5-a783-903135a046e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.831565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f623bd-b547-4ac5-a783-903135a046e1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.839987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f623bd-b547-4ac5-a783-903135a046e1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.841782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f623bd-b547-4ac5-a783-903135a046e1 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-73f623bd-b547-4ac5-a783-903135a046e1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:28.857247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:28.858285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:28.859769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 319/639] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:28.861372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe80900-174e-45fc-95d1-d2ce309f74b4 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.861372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe80900-174e-45fc-95d1-d2ce309f74b4 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.861372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe80900-174e-45fc-95d1-d2ce309f74b4 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.862770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 320/640] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:28.863961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25b4be-2e4e-4fab-a940-43434d988683 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.864251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25b4be-2e4e-4fab-a940-43434d988683 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.864552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25b4be-2e4e-4fab-a940-43434d988683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.883155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-af25b4be-2e4e-4fab-a940-43434d988683 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:28.884290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-af25b4be-2e4e-4fab-a940-43434d988683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:28.896940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.913597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150d807-c418-4459-9d10-69e9b79a7f23 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:28.947290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:28.952905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed944be1-6496-4fb7-b668-ff804c956f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:28.953902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed944be1-6496-4fb7-b668-ff804c956f6c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:28.954250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed944be1-6496-4fb7-b668-ff804c956f6c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:28.954591 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed944be1-6496-4fb7-b668-ff804c956f6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:28.962025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed944be1-6496-4fb7-b668-ff804c956f6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:28.963862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed944be1-6496-4fb7-b668-ff804c956f6c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-ed944be1-6496-4fb7-b668-ff804c956f6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:28.978739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:28.979950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:28.981001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:28.981001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:28.981001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:28.981001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:28.981001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:28.981001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:28.981001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:28.981001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:28.981001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 321/641] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:28.981643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed944be1-6496-4fb7-b668-ff804c956f6c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.010040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 321/642] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:29.010040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82064a5-e325-4cd0-bec4-6cf4489fc53d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.010040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82064a5-e325-4cd0-bec4-6cf4489fc53d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.010040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82064a5-e325-4cd0-bec4-6cf4489fc53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.030645 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-f82064a5-e325-4cd0-bec4-6cf4489fc53d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:29.031825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f82064a5-e325-4cd0-bec4-6cf4489fc53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:29.097955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 322/643] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:29.098954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82064a5-e325-4cd0-bec4-6cf4489fc53d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.099234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed944be1-6496-4fb7-b668-ff804c956f6c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.140339 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f9952-4ff6-49a2-82b1-211fe32599d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.140982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9952-4ff6-49a2-82b1-211fe32599d5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.141267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9952-4ff6-49a2-82b1-211fe32599d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.141545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f9952-4ff6-49a2-82b1-211fe32599d5 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.142454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f9952-4ff6-49a2-82b1-211fe32599d5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:29.142789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f9952-4ff6-49a2-82b1-211fe32599d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.144217 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f9952-4ff6-49a2-82b1-211fe32599d5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.163466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 322/644] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 23 15:37:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:29.164570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f9952-4ff6-49a2-82b1-211fe32599d5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.180489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:29.210954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82064a5-e325-4cd0-bec4-6cf4489fc53d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.219744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:29.246787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a0e5d-1e40-476e-92ad-3950c5f988d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.247589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a0e5d-1e40-476e-92ad-3950c5f988d0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.247968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a0e5d-1e40-476e-92ad-3950c5f988d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.248252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a0e5d-1e40-476e-92ad-3950c5f988d0 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.249221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a0e5d-1e40-476e-92ad-3950c5f988d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:29.249533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a0e5d-1e40-476e-92ad-3950c5f988d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.250963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a0e5d-1e40-476e-92ad-3950c5f988d0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.264576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 323/645] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:29.265551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a0e5d-1e40-476e-92ad-3950c5f988d0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.276083 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f9952-4ff6-49a2-82b1-211fe32599d5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.313462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a649e7-38fc-4594-956a-51a5b1925759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.314263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a649e7-38fc-4594-956a-51a5b1925759 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.314544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a649e7-38fc-4594-956a-51a5b1925759 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.314817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a649e7-38fc-4594-956a-51a5b1925759 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.325896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a649e7-38fc-4594-956a-51a5b1925759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-c4a649e7-38fc-4594-956a-51a5b1925759 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:29.333785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:29.334907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:29.335999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 323/646] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:29.336741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a649e7-38fc-4594-956a-51a5b1925759 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.388778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a0e5d-1e40-476e-92ad-3950c5f988d0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.422935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:29.428584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ee504-03a7-4168-81e5-a3f0ffac5677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.429515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ee504-03a7-4168-81e5-a3f0ffac5677 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.429870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ee504-03a7-4168-81e5-a3f0ffac5677 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.430216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ee504-03a7-4168-81e5-a3f0ffac5677 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.440612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a649e7-38fc-4594-956a-51a5b1925759 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.441851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ee504-03a7-4168-81e5-a3f0ffac5677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-d00ee504-03a7-4168-81e5-a3f0ffac5677 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:29.450217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:29.451355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:29.452957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:29.453753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:29.454305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 324/647] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:29.459698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ee504-03a7-4168-81e5-a3f0ffac5677 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.480929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e99705-0d22-451f-92b0-6a92caa61fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.481700 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e99705-0d22-451f-92b0-6a92caa61fa8 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.482051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e99705-0d22-451f-92b0-6a92caa61fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.482396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e99705-0d22-451f-92b0-6a92caa61fa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.555518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e99705-0d22-451f-92b0-6a92caa61fa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.557247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e99705-0d22-451f-92b0-6a92caa61fa8 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.567661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ee504-03a7-4168-81e5-a3f0ffac5677 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.603314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310b894-58e8-4240-ab79-0fb3b0b8343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.604099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310b894-58e8-4240-ab79-0fb3b0b8343f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.604414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310b894-58e8-4240-ab79-0fb3b0b8343f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.604702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310b894-58e8-4240-ab79-0fb3b0b8343f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.605540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310b894-58e8-4240-ab79-0fb3b0b8343f None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=b9129f0b362a4945a0eca7cc33571abd) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:29.605860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310b894-58e8-4240-ab79-0fb3b0b8343f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.607525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310b894-58e8-4240-ab79-0fb3b0b8343f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.620560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 325/648] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/role_assignments?user.id=b9129f0b362a4945a0eca7cc33571abd&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:29.621685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310b894-58e8-4240-ab79-0fb3b0b8343f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.671919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 324/649] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:29.673257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e99705-0d22-451f-92b0-6a92caa61fa8 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.686039 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:29.688771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:29.735666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310b894-58e8-4240-ab79-0fb3b0b8343f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.774596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c83bc-9b7d-457d-b95b-e3961354943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.775457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c83bc-9b7d-457d-b95b-e3961354943c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.775844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c83bc-9b7d-457d-b95b-e3961354943c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.776219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c83bc-9b7d-457d-b95b-e3961354943c None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.777133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c83bc-9b7d-457d-b95b-e3961354943c None admin] RBAC: Adding query filter params (name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:29.777518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c83bc-9b7d-457d-b95b-e3961354943c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.779030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c83bc-9b7d-457d-b95b-e3961354943c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.788756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e99705-0d22-451f-92b0-6a92caa61fa8 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.789264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 326/650] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:29.790296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c83bc-9b7d-457d-b95b-e3961354943c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.828942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec3d05-105f-46d2-b93b-56966138e22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.829745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3d05-105f-46d2-b93b-56966138e22c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.830026 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3d05-105f-46d2-b93b-56966138e22c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.830301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3d05-105f-46d2-b93b-56966138e22c None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.831051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec3d05-105f-46d2-b93b-56966138e22c None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:29.831360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec3d05-105f-46d2-b93b-56966138e22c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.832750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec3d05-105f-46d2-b93b-56966138e22c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.846263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 325/651] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:29.846263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec3d05-105f-46d2-b93b-56966138e22c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:29.904572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c83bc-9b7d-457d-b95b-e3961354943c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.927289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:29.934708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:29.950242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec72f9-e68a-4a1e-b3d4-8339457289ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:29.951369 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec72f9-e68a-4a1e-b3d4-8339457289ad None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:29.951901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec72f9-e68a-4a1e-b3d4-8339457289ad None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:29.953076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec72f9-e68a-4a1e-b3d4-8339457289ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:29.967143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec72f9-e68a-4a1e-b3d4-8339457289ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:29.968914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec72f9-e68a-4a1e-b3d4-8339457289ad None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:29.976565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec3d05-105f-46d2-b93b-56966138e22c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-02ec72f9-e68a-4a1e-b3d4-8339457289ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:29.992549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:29.993483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:29.994381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:29.994381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:29.994381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:29.994381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:29.996503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 327/652] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:29.997764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec72f9-e68a-4a1e-b3d4-8339457289ad None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.019961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae47174-0700-43c3-8f32-00bc4369242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.021147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae47174-0700-43c3-8f32-00bc4369242d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.021502 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae47174-0700-43c3-8f32-00bc4369242d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.021979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae47174-0700-43c3-8f32-00bc4369242d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.035705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae47174-0700-43c3-8f32-00bc4369242d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.038058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae47174-0700-43c3-8f32-00bc4369242d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-cae47174-0700-43c3-8f32-00bc4369242d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:30.059774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:30.060776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:30.062199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:30.062199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:30.062199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:30.062199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:30.062515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 326/653] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:30.066893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae47174-0700-43c3-8f32-00bc4369242d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.123835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec72f9-e68a-4a1e-b3d4-8339457289ad None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.164495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ea0f1-fba5-4cda-91aa-36d045c8add6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.165438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ea0f1-fba5-4cda-91aa-36d045c8add6 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.165724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ea0f1-fba5-4cda-91aa-36d045c8add6 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.166014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ea0f1-fba5-4cda-91aa-36d045c8add6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.177474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae47174-0700-43c3-8f32-00bc4369242d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.177938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ea0f1-fba5-4cda-91aa-36d045c8add6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.179437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ea0f1-fba5-4cda-91aa-36d045c8add6 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-bb5ea0f1-fba5-4cda-91aa-36d045c8add6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:30.199124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:30.201131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:30.203931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:30.203931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:30.203931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:30.203931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:30.212417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:30.212662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 328/654] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:30.212662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ea0f1-fba5-4cda-91aa-36d045c8add6 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.214454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fdf40-8004-417a-a944-a3bc7a087ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.215214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fdf40-8004-417a-a944-a3bc7a087ae8 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.215565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fdf40-8004-417a-a944-a3bc7a087ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.215910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fdf40-8004-417a-a944-a3bc7a087ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.216748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fdf40-8004-417a-a944-a3bc7a087ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:30.217124 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fdf40-8004-417a-a944-a3bc7a087ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.218660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fdf40-8004-417a-a944-a3bc7a087ae8 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.231112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 327/655] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:30.232035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fdf40-8004-417a-a944-a3bc7a087ae8 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.237609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:30.319408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ea0f1-fba5-4cda-91aa-36d045c8add6 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.352577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fdf40-8004-417a-a944-a3bc7a087ae8 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.354187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66d4b7-4ccf-4bfd-85f3-4af307affe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.355015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66d4b7-4ccf-4bfd-85f3-4af307affe83 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.355537 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66d4b7-4ccf-4bfd-85f3-4af307affe83 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.356006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66d4b7-4ccf-4bfd-85f3-4af307affe83 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.356955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66d4b7-4ccf-4bfd-85f3-4af307affe83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:30.357459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66d4b7-4ccf-4bfd-85f3-4af307affe83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.359654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66d4b7-4ccf-4bfd-85f3-4af307affe83 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.371670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 329/656] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:30.372834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66d4b7-4ccf-4bfd-85f3-4af307affe83 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.389273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c60562-d0ea-4b21-8632-cb7af0da124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.389976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c60562-d0ea-4b21-8632-cb7af0da124e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.390260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c60562-d0ea-4b21-8632-cb7af0da124e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.390536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c60562-d0ea-4b21-8632-cb7af0da124e None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.391380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c60562-d0ea-4b21-8632-cb7af0da124e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:30.391694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c60562-d0ea-4b21-8632-cb7af0da124e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.393102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c60562-d0ea-4b21-8632-cb7af0da124e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.410051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 328/657] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:30.410051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c60562-d0ea-4b21-8632-cb7af0da124e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.472301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:30.478079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:30.496687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66d4b7-4ccf-4bfd-85f3-4af307affe83 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.533665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625362d3-604f-4bf9-8e61-a76d1d7a0c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.534606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625362d3-604f-4bf9-8e61-a76d1d7a0c6f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.534839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625362d3-604f-4bf9-8e61-a76d1d7a0c6f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.535121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625362d3-604f-4bf9-8e61-a76d1d7a0c6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.543865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625362d3-604f-4bf9-8e61-a76d1d7a0c6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.545351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625362d3-604f-4bf9-8e61-a76d1d7a0c6f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.547487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c60562-d0ea-4b21-8632-cb7af0da124e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-625362d3-604f-4bf9-8e61-a76d1d7a0c6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:30.560744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:30.562454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:30.563500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:30.563500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:30.563500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:30.563500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:30.563500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:30.563500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:30.563500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:30.563500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:30.563500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 330/658] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:30.563500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625362d3-604f-4bf9-8e61-a76d1d7a0c6f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.591885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55611c4f-9d53-48cc-9ab3-ae408e74ec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.593241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55611c4f-9d53-48cc-9ab3-ae408e74ec0b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.593523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55611c4f-9d53-48cc-9ab3-ae408e74ec0b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.593798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55611c4f-9d53-48cc-9ab3-ae408e74ec0b None admin] PATH_INFO: `/v3/users/glance` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.607670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55611c4f-9d53-48cc-9ab3-ae408e74ec0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.610372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55611c4f-9d53-48cc-9ab3-ae408e74ec0b None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-55611c4f-9d53-48cc-9ab3-ae408e74ec0b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:30.630273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:30.631409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:30.632583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:30.632583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:30.632583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:30.632583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:30.632583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:30.632583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:30.632583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:30.632583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:30.632583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 15:37:30.632583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:30.639305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 329/659] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:30.639305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55611c4f-9d53-48cc-9ab3-ae408e74ec0b None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.683997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625362d3-604f-4bf9-8e61-a76d1d7a0c6f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.721368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbde81-490b-45d0-8dfa-217ef3ee5785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.722377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbde81-490b-45d0-8dfa-217ef3ee5785 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.722704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbde81-490b-45d0-8dfa-217ef3ee5785 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.723081 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbde81-490b-45d0-8dfa-217ef3ee5785 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.731410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbde81-490b-45d0-8dfa-217ef3ee5785 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.733335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbde81-490b-45d0-8dfa-217ef3ee5785 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-eadbde81-490b-45d0-8dfa-217ef3ee5785 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:30.749756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:30.750863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:30.751789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:30.751789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:30.751789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:30.751789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:30.751789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:30.751789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:30.751789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:30.751789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:30.751789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 15:37:30.751789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:30.755371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55611c4f-9d53-48cc-9ab3-ae408e74ec0b None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.756329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 331/660] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:30.757410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbde81-490b-45d0-8dfa-217ef3ee5785 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.759223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:30.761553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:30.792172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523160f-c705-4acf-9599-7d1a5b96c609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.793292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523160f-c705-4acf-9599-7d1a5b96c609 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.793773 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523160f-c705-4acf-9599-7d1a5b96c609 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.794257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523160f-c705-4acf-9599-7d1a5b96c609 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.795250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523160f-c705-4acf-9599-7d1a5b96c609 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:30.795761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523160f-c705-4acf-9599-7d1a5b96c609 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.798111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523160f-c705-4acf-9599-7d1a5b96c609 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.815504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 330/661] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:30.816687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523160f-c705-4acf-9599-7d1a5b96c609 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.866991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbde81-490b-45d0-8dfa-217ef3ee5785 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.906041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4c97c-b4fc-440b-860a-7af6943511d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.906836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4c97c-b4fc-440b-860a-7af6943511d2 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.907049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4c97c-b4fc-440b-860a-7af6943511d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.907334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4c97c-b4fc-440b-860a-7af6943511d2 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.908322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4c97c-b4fc-440b-860a-7af6943511d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:30.908683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4c97c-b4fc-440b-860a-7af6943511d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.910315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4c97c-b4fc-440b-860a-7af6943511d2 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.931424 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523160f-c705-4acf-9599-7d1a5b96c609 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:30.936820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 332/662] 10.222.0.23 () {56 vars in 1137 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:30.936820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4c97c-b4fc-440b-860a-7af6943511d2 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:30.973075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27a1c1-de47-4520-8503-3417dc460868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:30.974131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27a1c1-de47-4520-8503-3417dc460868 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:30.974611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27a1c1-de47-4520-8503-3417dc460868 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:30.975084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27a1c1-de47-4520-8503-3417dc460868 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:30.976131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27a1c1-de47-4520-8503-3417dc460868 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:30.976660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27a1c1-de47-4520-8503-3417dc460868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:30.978294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27a1c1-de47-4520-8503-3417dc460868 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:30.997028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 331/663] 10.222.0.23 () {56 vars in 1139 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:30.998169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27a1c1-de47-4520-8503-3417dc460868 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.023264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.031475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.051997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4c97c-b4fc-440b-860a-7af6943511d2 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.094180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b57a5-defa-423e-b03f-4af40c9cca5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.095241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b57a5-defa-423e-b03f-4af40c9cca5f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.095844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b57a5-defa-423e-b03f-4af40c9cca5f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.096333 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b57a5-defa-423e-b03f-4af40c9cca5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.097277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b57a5-defa-423e-b03f-4af40c9cca5f None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=0d30c8d1f883400ba397ea09d42a0aa8) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:31.098211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b57a5-defa-423e-b03f-4af40c9cca5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.099990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b57a5-defa-423e-b03f-4af40c9cca5f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.105376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27a1c1-de47-4520-8503-3417dc460868 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.106335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 333/664] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/role_assignments?user.id=0d30c8d1f883400ba397ea09d42a0aa8&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:31.107175 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b57a5-defa-423e-b03f-4af40c9cca5f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.145544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc8c70-ed01-4c15-a4f5-594d88736f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.146517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8c70-ed01-4c15-a4f5-594d88736f28 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.146977 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8c70-ed01-4c15-a4f5-594d88736f28 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.147428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8c70-ed01-4c15-a4f5-594d88736f28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.231783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b57a5-defa-423e-b03f-4af40c9cca5f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.248947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc8c70-ed01-4c15-a4f5-594d88736f28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.250500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc8c70-ed01-4c15-a4f5-594d88736f28 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.261865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.275217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423b3a0-c4a7-4231-ac74-1c4b97473378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.276221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423b3a0-c4a7-4231-ac74-1c4b97473378 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.276774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423b3a0-c4a7-4231-ac74-1c4b97473378 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.277252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423b3a0-c4a7-4231-ac74-1c4b97473378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.288042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.362369 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423b3a0-c4a7-4231-ac74-1c4b97473378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.363817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423b3a0-c4a7-4231-ac74-1c4b97473378 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.382892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 332/665] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:31.384062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bce9eb-80c0-46a0-821a-0fabb026bf7b None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.384334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bce9eb-80c0-46a0-821a-0fabb026bf7b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.388705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bce9eb-80c0-46a0-821a-0fabb026bf7b None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.390742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 333/666] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:31.392112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bce9eb-80c0-46a0-821a-0fabb026bf7b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.497541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 334/667] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:31.499657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7665fd-dd52-4b37-8790-6cc0b94cd6fa None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.500533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7665fd-dd52-4b37-8790-6cc0b94cd6fa None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.500824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7665fd-dd52-4b37-8790-6cc0b94cd6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.518768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bce9eb-80c0-46a0-821a-0fabb026bf7b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.524139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2f7665fd-dd52-4b37-8790-6cc0b94cd6fa None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:31.525585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2f7665fd-dd52-4b37-8790-6cc0b94cd6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:31.546111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.551879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.561540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266e756-bc9c-42ea-9839-0fa2f38d8279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.566541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266e756-bc9c-42ea-9839-0fa2f38d8279 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.566541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266e756-bc9c-42ea-9839-0fa2f38d8279 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.566541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266e756-bc9c-42ea-9839-0fa2f38d8279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.577110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266e756-bc9c-42ea-9839-0fa2f38d8279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.579143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266e756-bc9c-42ea-9839-0fa2f38d8279 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-a266e756-bc9c-42ea-9839-0fa2f38d8279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:31.602465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:31.603393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:31.604271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:31.604271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:31.604271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:31.604271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:31.608437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 334/668] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:31.610141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266e756-bc9c-42ea-9839-0fa2f38d8279 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.624942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 335/669] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:31.626499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7665fd-dd52-4b37-8790-6cc0b94cd6fa None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.718256 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266e756-bc9c-42ea-9839-0fa2f38d8279 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.754417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7665fd-dd52-4b37-8790-6cc0b94cd6fa None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.754856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e3a6-2620-45ba-8b03-df3729aec9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.755714 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e3a6-2620-45ba-8b03-df3729aec9fc None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.755998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e3a6-2620-45ba-8b03-df3729aec9fc None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.756268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e3a6-2620-45ba-8b03-df3729aec9fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.769213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e3a6-2620-45ba-8b03-df3729aec9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.770562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e3a6-2620-45ba-8b03-df3729aec9fc None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-5bb7e3a6-2620-45ba-8b03-df3729aec9fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:31.789538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:31.790453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:31.791354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:31.791354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:31.791354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:31.791354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:31.791354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 335/670] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:31.792050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e3a6-2620-45ba-8b03-df3729aec9fc None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.804398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afd449-4193-46e4-8700-27c284c8d962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.804398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afd449-4193-46e4-8700-27c284c8d962 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.804398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afd449-4193-46e4-8700-27c284c8d962 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.804398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afd449-4193-46e4-8700-27c284c8d962 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.804811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afd449-4193-46e4-8700-27c284c8d962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:31.804811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afd449-4193-46e4-8700-27c284c8d962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.804811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afd449-4193-46e4-8700-27c284c8d962 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:31.821325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 336/671] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:31.822455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afd449-4193-46e4-8700-27c284c8d962 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.825600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.840677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:31.904807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e3a6-2620-45ba-8b03-df3729aec9fc None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.941793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22684100-ed83-4bce-bcc8-71d7c5af75ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.942599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22684100-ed83-4bce-bcc8-71d7c5af75ab None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.942878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22684100-ed83-4bce-bcc8-71d7c5af75ab None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.943151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22684100-ed83-4bce-bcc8-71d7c5af75ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.944649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afd449-4193-46e4-8700-27c284c8d962 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:31.955409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22684100-ed83-4bce-bcc8-71d7c5af75ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-22684100-ed83-4bce-bcc8-71d7c5af75ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:31.965725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:31.966713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:31.967538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:31.968282 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 336/672] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:31.968846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22684100-ed83-4bce-bcc8-71d7c5af75ab None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:31.980855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833db791-304b-407f-aa77-e6448bd4969b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:31.981535 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833db791-304b-407f-aa77-e6448bd4969b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:31.981928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833db791-304b-407f-aa77-e6448bd4969b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:31.982216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833db791-304b-407f-aa77-e6448bd4969b None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:31.982949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833db791-304b-407f-aa77-e6448bd4969b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:31.983367 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833db791-304b-407f-aa77-e6448bd4969b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:31.985339 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833db791-304b-407f-aa77-e6448bd4969b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.002447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 337/673] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:32.003335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833db791-304b-407f-aa77-e6448bd4969b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.072874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.085697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.087164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22684100-ed83-4bce-bcc8-71d7c5af75ab None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.111738 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833db791-304b-407f-aa77-e6448bd4969b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.123486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9b798-dbd0-4a6f-a4b4-a659c797d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.124849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9b798-dbd0-4a6f-a4b4-a659c797d3a8 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.125143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9b798-dbd0-4a6f-a4b4-a659c797d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.125416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9b798-dbd0-4a6f-a4b4-a659c797d3a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.132142 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9b798-dbd0-4a6f-a4b4-a659c797d3a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.133610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9b798-dbd0-4a6f-a4b4-a659c797d3a8 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.147700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fedfd-01d9-45d3-9264-6115093a4c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-5ee9b798-dbd0-4a6f-a4b4-a659c797d3a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:32.147951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:32.148837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:32.154794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:32.154794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:32.154794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:32.154794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:32.154794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:32.154794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:32.154794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:32.154794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:32.155315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fedfd-01d9-45d3-9264-6115093a4c7d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.155315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fedfd-01d9-45d3-9264-6115093a4c7d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.155315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fedfd-01d9-45d3-9264-6115093a4c7d None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.156069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 337/674] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:32.156138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fedfd-01d9-45d3-9264-6115093a4c7d None admin] RBAC: Adding query filter params (name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:32.156138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fedfd-01d9-45d3-9264-6115093a4c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.156271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9b798-dbd0-4a6f-a4b4-a659c797d3a8 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.156344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fedfd-01d9-45d3-9264-6115093a4c7d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.164763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 338/675] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:32.170214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87319f6f-91d8-4e37-80c0-5704229db535 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.170513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87319f6f-91d8-4e37-80c0-5704229db535 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.171817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87319f6f-91d8-4e37-80c0-5704229db535 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.173528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 339/676] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:32.177716 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87319f6f-91d8-4e37-80c0-5704229db535 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.263199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9b798-dbd0-4a6f-a4b4-a659c797d3a8 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.287928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87319f6f-91d8-4e37-80c0-5704229db535 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.304320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40358ba4-8313-4750-b3b3-fa8816d71ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.305149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40358ba4-8313-4750-b3b3-fa8816d71ea4 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.305432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40358ba4-8313-4750-b3b3-fa8816d71ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.305707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40358ba4-8313-4750-b3b3-fa8816d71ea4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.313164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40358ba4-8313-4750-b3b3-fa8816d71ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.314641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40358ba4-8313-4750-b3b3-fa8816d71ea4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.327571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775abde8-a020-4f0d-aa27-af179f517997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.328153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775abde8-a020-4f0d-aa27-af179f517997 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.328464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775abde8-a020-4f0d-aa27-af179f517997 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.328741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775abde8-a020-4f0d-aa27-af179f517997 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.329525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775abde8-a020-4f0d-aa27-af179f517997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-40358ba4-8313-4750-b3b3-fa8816d71ea4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:32.329770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:32.330631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:32.331541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:32.331541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:32.331541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:32.331541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:32.331541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:32.331541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:32.331541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:32.331541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:32.331969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775abde8-a020-4f0d-aa27-af179f517997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.332039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 338/677] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:32.332106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775abde8-a020-4f0d-aa27-af179f517997 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.332172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40358ba4-8313-4750-b3b3-fa8816d71ea4 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.334659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.345404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 340/678] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:32.346553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.351502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fa08f-2356-46cb-93c1-af4e8bc8c583 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.351794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fa08f-2356-46cb-93c1-af4e8bc8c583 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.352197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fa08f-2356-46cb-93c1-af4e8bc8c583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.370149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-f95fa08f-2356-46cb-93c1-af4e8bc8c583 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:32.371329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f95fa08f-2356-46cb-93c1-af4e8bc8c583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:32.441781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 341/679] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:32.442508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fa08f-2356-46cb-93c1-af4e8bc8c583 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.462834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40358ba4-8313-4750-b3b3-fa8816d71ea4 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.501745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2307d-d716-42d6-b2f4-1c17dfbd984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.502538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2307d-d716-42d6-b2f4-1c17dfbd984b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.502823 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2307d-d716-42d6-b2f4-1c17dfbd984b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.503097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2307d-d716-42d6-b2f4-1c17dfbd984b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.515430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2307d-d716-42d6-b2f4-1c17dfbd984b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.516911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2307d-d716-42d6-b2f4-1c17dfbd984b None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-8ff2307d-d716-42d6-b2f4-1c17dfbd984b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:32.534821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:32.535941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:32.536886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:32.536886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:32.536886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:32.536886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:32.536886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 339/680] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:32.536886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2307d-d716-42d6-b2f4-1c17dfbd984b None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.550983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fa08f-2356-46cb-93c1-af4e8bc8c583 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.578914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.588979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49e953-b13c-43c3-9865-2c4fc5b25a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.602912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49e953-b13c-43c3-9865-2c4fc5b25a97 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.602912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49e953-b13c-43c3-9865-2c4fc5b25a97 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.602912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49e953-b13c-43c3-9865-2c4fc5b25a97 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.602912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49e953-b13c-43c3-9865-2c4fc5b25a97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:32.602912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49e953-b13c-43c3-9865-2c4fc5b25a97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.602912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49e953-b13c-43c3-9865-2c4fc5b25a97 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.603374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.610233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 342/681] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:32.610233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49e953-b13c-43c3-9865-2c4fc5b25a97 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.644157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2307d-d716-42d6-b2f4-1c17dfbd984b None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.679245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfeb781-dc29-4a0c-8516-8c6443be4fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.679903 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeb781-dc29-4a0c-8516-8c6443be4fe5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.680189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeb781-dc29-4a0c-8516-8c6443be4fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.680496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfeb781-dc29-4a0c-8516-8c6443be4fe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.681331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeb781-dc29-4a0c-8516-8c6443be4fe5 None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=142e6e44d6544f6a891ed3f913de74ae) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:32.681647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeb781-dc29-4a0c-8516-8c6443be4fe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.683470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfeb781-dc29-4a0c-8516-8c6443be4fe5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.690476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 340/682] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/role_assignments?user.id=142e6e44d6544f6a891ed3f913de74ae&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:32.691353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfeb781-dc29-4a0c-8516-8c6443be4fe5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.714985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49e953-b13c-43c3-9865-2c4fc5b25a97 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.761577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead0e73-845a-4e51-966c-3e0feb08dd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.762437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead0e73-845a-4e51-966c-3e0feb08dd67 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.762721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead0e73-845a-4e51-966c-3e0feb08dd67 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.762999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead0e73-845a-4e51-966c-3e0feb08dd67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.778787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead0e73-845a-4e51-966c-3e0feb08dd67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-8ead0e73-845a-4e51-966c-3e0feb08dd67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:32.789926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:32.790963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:32.791793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:32.796545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 343/683] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:32.797822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead0e73-845a-4e51-966c-3e0feb08dd67 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.845809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfeb781-dc29-4a0c-8516-8c6443be4fe5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.865833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.877848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:32.893513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39da688-6db2-4bb4-abc7-16c188d6ae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.894713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39da688-6db2-4bb4-abc7-16c188d6ae2f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.894713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39da688-6db2-4bb4-abc7-16c188d6ae2f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.894713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39da688-6db2-4bb4-abc7-16c188d6ae2f None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.895621 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39da688-6db2-4bb4-abc7-16c188d6ae2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:32.895930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39da688-6db2-4bb4-abc7-16c188d6ae2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.898864 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39da688-6db2-4bb4-abc7-16c188d6ae2f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.920641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 341/684] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:32.920641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39da688-6db2-4bb4-abc7-16c188d6ae2f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:32.941022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead0e73-845a-4e51-966c-3e0feb08dd67 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:32.983096 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36b794-f768-4b87-a2e3-7e87427a2b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:32.983774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36b794-f768-4b87-a2e3-7e87427a2b38 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:32.984122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36b794-f768-4b87-a2e3-7e87427a2b38 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:32.984446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36b794-f768-4b87-a2e3-7e87427a2b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:32.985289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36b794-f768-4b87-a2e3-7e87427a2b38 None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=a39f5189d63d467dbd914f553c261c01) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:32.985601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36b794-f768-4b87-a2e3-7e87427a2b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:32.987044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36b794-f768-4b87-a2e3-7e87427a2b38 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:32.994838 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 344/685] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/role_assignments?user.id=a39f5189d63d467dbd914f553c261c01&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:32.995566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36b794-f768-4b87-a2e3-7e87427a2b38 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.038060 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39da688-6db2-4bb4-abc7-16c188d6ae2f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.076668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944530d-d9c3-471a-85ff-2136b8800352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.077996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944530d-d9c3-471a-85ff-2136b8800352 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.078285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944530d-d9c3-471a-85ff-2136b8800352 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.078564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944530d-d9c3-471a-85ff-2136b8800352 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.079389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944530d-d9c3-471a-85ff-2136b8800352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:33.079702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944530d-d9c3-471a-85ff-2136b8800352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.081133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944530d-d9c3-471a-85ff-2136b8800352 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.091540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 342/686] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:33.099364 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944530d-d9c3-471a-85ff-2136b8800352 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.148375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36b794-f768-4b87-a2e3-7e87427a2b38 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.167018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:33.171216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:33.229114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aa9ba-5296-4044-b020-73c717595b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.230099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aa9ba-5296-4044-b020-73c717595b64 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.231517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aa9ba-5296-4044-b020-73c717595b64 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.231827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aa9ba-5296-4044-b020-73c717595b64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.242265 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109aa9ba-5296-4044-b020-73c717595b64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.243524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944530d-d9c3-471a-85ff-2136b8800352 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.243710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109aa9ba-5296-4044-b020-73c717595b64 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-109aa9ba-5296-4044-b020-73c717595b64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:33.264683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:33.266257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:33.267184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 345/687] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:33.272353 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aa9ba-5296-4044-b020-73c717595b64 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.297988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18576f52-4070-44bf-8314-31a94a25187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.298924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18576f52-4070-44bf-8314-31a94a25187e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.299214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18576f52-4070-44bf-8314-31a94a25187e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.299487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18576f52-4070-44bf-8314-31a94a25187e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.313083 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18576f52-4070-44bf-8314-31a94a25187e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.314815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18576f52-4070-44bf-8314-31a94a25187e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-18576f52-4070-44bf-8314-31a94a25187e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:33.334371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:33.335362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:33.336239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:33.336239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:33.336239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:33.336239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:33.336239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 343/688] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:33.342392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18576f52-4070-44bf-8314-31a94a25187e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.404105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aa9ba-5296-4044-b020-73c717595b64 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.439573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0aafd-bb16-4214-b5ae-18c00a6e2992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.440251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0aafd-bb16-4214-b5ae-18c00a6e2992 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.440564 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0aafd-bb16-4214-b5ae-18c00a6e2992 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.440837 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0aafd-bb16-4214-b5ae-18c00a6e2992 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.441677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0aafd-bb16-4214-b5ae-18c00a6e2992 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:33.441984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0aafd-bb16-4214-b5ae-18c00a6e2992 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.443418 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0aafd-bb16-4214-b5ae-18c00a6e2992 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.461239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 346/689] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 23 15:37:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:33.466524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0aafd-bb16-4214-b5ae-18c00a6e2992 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.498116 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:33.498723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:33.514360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18576f52-4070-44bf-8314-31a94a25187e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.559878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b65e83-82e2-4b10-9924-c4dad875034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.560581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65e83-82e2-4b10-9924-c4dad875034d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.560869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65e83-82e2-4b10-9924-c4dad875034d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.561310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65e83-82e2-4b10-9924-c4dad875034d None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.563025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b65e83-82e2-4b10-9924-c4dad875034d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:33.563459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b65e83-82e2-4b10-9924-c4dad875034d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.565163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b65e83-82e2-4b10-9924-c4dad875034d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.581807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 344/690] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:33.582443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0aafd-bb16-4214-b5ae-18c00a6e2992 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.583136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7620da-6fa4-4e20-8612-eb9d1a26ea4e None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.583428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7620da-6fa4-4e20-8612-eb9d1a26ea4e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.584423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7620da-6fa4-4e20-8612-eb9d1a26ea4e None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.588436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 345/691] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:33.589531 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7620da-6fa4-4e20-8612-eb9d1a26ea4e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.625237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d4690-b210-4932-a601-9933732b69db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.625901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d4690-b210-4932-a601-9933732b69db None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.626182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d4690-b210-4932-a601-9933732b69db None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.626454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d4690-b210-4932-a601-9933732b69db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.703731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d4690-b210-4932-a601-9933732b69db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.705131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d4690-b210-4932-a601-9933732b69db None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.724656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7620da-6fa4-4e20-8612-eb9d1a26ea4e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:33.740903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:33.760584 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71181007-37e1-44da-9b3a-53384663c6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:33.761751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71181007-37e1-44da-9b3a-53384663c6e1 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.762039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71181007-37e1-44da-9b3a-53384663c6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.762317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71181007-37e1-44da-9b3a-53384663c6e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.770592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71181007-37e1-44da-9b3a-53384663c6e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:33.772034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71181007-37e1-44da-9b3a-53384663c6e1 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-71181007-37e1-44da-9b3a-53384663c6e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:33.787238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:33.789562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 346/692] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 23 15:37:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f92da-7bf0-4762-848d-b72bdbda9826 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:33.791814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f92da-7bf0-4762-848d-b72bdbda9826 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:33.793777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f92da-7bf0-4762-848d-b72bdbda9826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:33.809371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ed4f92da-7bf0-4762-848d-b72bdbda9826 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:33.810678 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ed4f92da-7bf0-4762-848d-b72bdbda9826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:33.822872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:33.834676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 347/693] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:33.838911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d4690-b210-4932-a601-9933732b69db None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.896493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 347/694] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:33.897408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f92da-7bf0-4762-848d-b72bdbda9826 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:33.969181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d4690-b210-4932-a601-9933732b69db None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.007059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:34.014471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f92da-7bf0-4762-848d-b72bdbda9826 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.019616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c19faa-5fda-4e7a-b16b-75a008ef5b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.020554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c19faa-5fda-4e7a-b16b-75a008ef5b2c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.022652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c19faa-5fda-4e7a-b16b-75a008ef5b2c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.022652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c19faa-5fda-4e7a-b16b-75a008ef5b2c None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.022652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c19faa-5fda-4e7a-b16b-75a008ef5b2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:34.022652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c19faa-5fda-4e7a-b16b-75a008ef5b2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.024773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c19faa-5fda-4e7a-b16b-75a008ef5b2c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.051610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 348/695] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 23 15:37:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:34.053141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c19faa-5fda-4e7a-b16b-75a008ef5b2c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.053691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4027a9f-293f-4a9b-8251-b2d05b043cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.054588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4027a9f-293f-4a9b-8251-b2d05b043cdd None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.054876 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4027a9f-293f-4a9b-8251-b2d05b043cdd None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.055156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4027a9f-293f-4a9b-8251-b2d05b043cdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.066883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4027a9f-293f-4a9b-8251-b2d05b043cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.068350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4027a9f-293f-4a9b-8251-b2d05b043cdd None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.076042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-a4027a9f-293f-4a9b-8251-b2d05b043cdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:34.088581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:34.097726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:34.098622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:34.098622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:34.098622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:34.098622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:34.098622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 348/696] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:34.098622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4027a9f-293f-4a9b-8251-b2d05b043cdd None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.183174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c19faa-5fda-4e7a-b16b-75a008ef5b2c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.211640 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4027a9f-293f-4a9b-8251-b2d05b043cdd None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.229523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5dc6c-5c45-431e-8d90-def58d844346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.230926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5dc6c-5c45-431e-8d90-def58d844346 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.231361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5dc6c-5c45-431e-8d90-def58d844346 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.231798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5dc6c-5c45-431e-8d90-def58d844346 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.232637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5dc6c-5c45-431e-8d90-def58d844346 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.234738 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5dc6c-5c45-431e-8d90-def58d844346 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-e1a5dc6c-5c45-431e-8d90-def58d844346 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:34.251604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:34.252652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:37:34.253481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:34.256158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbda797-60a6-43ad-a747-6d863cda824b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.257040 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbda797-60a6-43ad-a747-6d863cda824b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.257489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbda797-60a6-43ad-a747-6d863cda824b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.257925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbda797-60a6-43ad-a747-6d863cda824b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.258027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 349/697] 10.222.0.23 () {56 vars in 1129 bytes} [Mon May 23 15:37:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:34.258891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5dc6c-5c45-431e-8d90-def58d844346 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.314788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:34.345813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:34.350738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbda797-60a6-43ad-a747-6d863cda824b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.352410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbda797-60a6-43ad-a747-6d863cda824b None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.384633 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5dc6c-5c45-431e-8d90-def58d844346 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.430290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a73937-0bc1-45fa-9648-4d88014d712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.431232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a73937-0bc1-45fa-9648-4d88014d712f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.431676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a73937-0bc1-45fa-9648-4d88014d712f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.432107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a73937-0bc1-45fa-9648-4d88014d712f None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.433541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a73937-0bc1-45fa-9648-4d88014d712f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:34.434019 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a73937-0bc1-45fa-9648-4d88014d712f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.436026 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a73937-0bc1-45fa-9648-4d88014d712f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.458794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 350/698] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:34.458794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a73937-0bc1-45fa-9648-4d88014d712f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.468209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 349/699] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:34.469276 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbda797-60a6-43ad-a747-6d863cda824b None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.566331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a73937-0bc1-45fa-9648-4d88014d712f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.578907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:34.596718 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:34.601997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbda797-60a6-43ad-a747-6d863cda824b None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.604652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c1724-3c3e-4684-9e03-1ceacc83616b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.605648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c1724-3c3e-4684-9e03-1ceacc83616b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.605913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c1724-3c3e-4684-9e03-1ceacc83616b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.606328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c1724-3c3e-4684-9e03-1ceacc83616b None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.607505 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c1724-3c3e-4684-9e03-1ceacc83616b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:34.608301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c1724-3c3e-4684-9e03-1ceacc83616b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.610329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c1724-3c3e-4684-9e03-1ceacc83616b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.620635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 351/700] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 23 15:37:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:34.621635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c1724-3c3e-4684-9e03-1ceacc83616b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.647246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb512a9-bae6-44e9-b6e1-90269d362128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.648577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb512a9-bae6-44e9-b6e1-90269d362128 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.649039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb512a9-bae6-44e9-b6e1-90269d362128 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.649714 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb512a9-bae6-44e9-b6e1-90269d362128 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.659302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb512a9-bae6-44e9-b6e1-90269d362128 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.662105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb512a9-bae6-44e9-b6e1-90269d362128 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-8cb512a9-bae6-44e9-b6e1-90269d362128 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:34.679345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:34.680481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:34.681664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:34.681664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:34.681664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:34.681664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:34.681664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:34.681664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:34.681664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:34.681664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:34.689680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 350/701] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:34.690428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb512a9-bae6-44e9-b6e1-90269d362128 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.740735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c1724-3c3e-4684-9e03-1ceacc83616b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.783352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ccd429-f6f5-414e-85b4-aac79d124bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.784287 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccd429-f6f5-414e-85b4-aac79d124bc5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.784669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccd429-f6f5-414e-85b4-aac79d124bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.785018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccd429-f6f5-414e-85b4-aac79d124bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.795852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ccd429-f6f5-414e-85b4-aac79d124bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.797958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ccd429-f6f5-414e-85b4-aac79d124bc5 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-d9ccd429-f6f5-414e-85b4-aac79d124bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:34.816882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:34.817801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:34.818685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:34.818685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:34.818685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:34.818685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:34.818685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 352/702] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:34.819803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ccd429-f6f5-414e-85b4-aac79d124bc5 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.837753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb512a9-bae6-44e9-b6e1-90269d362128 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.848835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:34.886460 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49023c-295f-46c5-8e3c-67bd8d0bf5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.888097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49023c-295f-46c5-8e3c-67bd8d0bf5a6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.888711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49023c-295f-46c5-8e3c-67bd8d0bf5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.889268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49023c-295f-46c5-8e3c-67bd8d0bf5a6 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.890189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49023c-295f-46c5-8e3c-67bd8d0bf5a6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:34.890675 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49023c-295f-46c5-8e3c-67bd8d0bf5a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.893119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49023c-295f-46c5-8e3c-67bd8d0bf5a6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.908543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 351/703] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 23 15:37:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:34.909704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49023c-295f-46c5-8e3c-67bd8d0bf5a6 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:34.934041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ccd429-f6f5-414e-85b4-aac79d124bc5 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:34.976123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c823e4e-0f66-4c5f-970a-799c6cf04bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:34.977189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c823e4e-0f66-4c5f-970a-799c6cf04bde None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:34.977659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c823e4e-0f66-4c5f-970a-799c6cf04bde None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:34.978109 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c823e4e-0f66-4c5f-970a-799c6cf04bde None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:34.979091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c823e4e-0f66-4c5f-970a-799c6cf04bde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:34.979582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c823e4e-0f66-4c5f-970a-799c6cf04bde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:34.981357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c823e4e-0f66-4c5f-970a-799c6cf04bde None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:34.996441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 353/704] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:35.003444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c823e4e-0f66-4c5f-970a-799c6cf04bde None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:35.111221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:35.130501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c823e4e-0f66-4c5f-970a-799c6cf04bde None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:35.178433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:35.185199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ce477-32cb-4326-a829-c167de8cfbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:35.186282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ce477-32cb-4326-a829-c167de8cfbb4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.186282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ce477-32cb-4326-a829-c167de8cfbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.186524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ce477-32cb-4326-a829-c167de8cfbb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.187510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ce477-32cb-4326-a829-c167de8cfbb4 None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=e57de80064b74ad59d33abb83a5d17f1) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:35.187731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ce477-32cb-4326-a829-c167de8cfbb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:35.189259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ce477-32cb-4326-a829-c167de8cfbb4 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:35.214810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 354/705] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:35 2022] GET /identity/v3/role_assignments?user.id=e57de80064b74ad59d33abb83a5d17f1&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:35.247288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66091be3-0fd2-4317-9775-bb2c555e5b05 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.247610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66091be3-0fd2-4317-9775-bb2c555e5b05 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.248007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66091be3-0fd2-4317-9775-bb2c555e5b05 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.250042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 355/706] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:35.256162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fab9b-18b6-4770-aa59-42fcb446f585 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.256677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fab9b-18b6-4770-aa59-42fcb446f585 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.257029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fab9b-18b6-4770-aa59-42fcb446f585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.275974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-dc0fab9b-18b6-4770-aa59-42fcb446f585 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:35.277474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-dc0fab9b-18b6-4770-aa59-42fcb446f585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:35.280310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49023c-295f-46c5-8e3c-67bd8d0bf5a6 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:35.328314 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1ffe4-0c7e-45e7-ad2d-468adc71ec8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:35.329428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1ffe4-0c7e-45e7-ad2d-468adc71ec8c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.329910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1ffe4-0c7e-45e7-ad2d-468adc71ec8c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.330381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1ffe4-0c7e-45e7-ad2d-468adc71ec8c None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.331314 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1ffe4-0c7e-45e7-ad2d-468adc71ec8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:35.331828 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1ffe4-0c7e-45e7-ad2d-468adc71ec8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:35.333509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1ffe4-0c7e-45e7-ad2d-468adc71ec8c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:35.348005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 352/707] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:35.365281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1ffe4-0c7e-45e7-ad2d-468adc71ec8c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:35.405042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 356/708] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:35.483316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fab9b-18b6-4770-aa59-42fcb446f585 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:35.487899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:35.511955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1ffe4-0c7e-45e7-ad2d-468adc71ec8c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:35.553448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1aa15-a9a4-4543-919b-40ccf0a7571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:35.554221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1aa15-a9a4-4543-919b-40ccf0a7571d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.554502 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1aa15-a9a4-4543-919b-40ccf0a7571d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.554774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1aa15-a9a4-4543-919b-40ccf0a7571d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.562476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1aa15-a9a4-4543-919b-40ccf0a7571d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:35.563569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1aa15-a9a4-4543-919b-40ccf0a7571d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-81a1aa15-a9a4-4543-919b-40ccf0a7571d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:35.579655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:35.580663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:35.581538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:35.581538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:35.581538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:35.581538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:35.581538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:35.581538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:35.581538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:35.581538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:35.581538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 353/709] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:35.582441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51638f68-a5e3-4293-8a97-a0dda2ac0997 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.582728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51638f68-a5e3-4293-8a97-a0dda2ac0997 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.582859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:35.583018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51638f68-a5e3-4293-8a97-a0dda2ac0997 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.588407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 354/710] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:35.589196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51638f68-a5e3-4293-8a97-a0dda2ac0997 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:35.604932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fab9b-18b6-4770-aa59-42fcb446f585 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:35.649656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c2a24-40d8-4776-a5fd-702220b0f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:35.650383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c2a24-40d8-4776-a5fd-702220b0f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.650670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c2a24-40d8-4776-a5fd-702220b0f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.650946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c2a24-40d8-4776-a5fd-702220b0f8f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.665354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c2a24-40d8-4776-a5fd-702220b0f8f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:35.667108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c2a24-40d8-4776-a5fd-702220b0f8f0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-9e9c2a24-40d8-4776-a5fd-702220b0f8f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:35.689409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:35.690348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:35.691251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:35.691251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:35.691251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:35.691251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:35.691251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 357/711] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:35.692449 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b5534-8b40-4904-9767-6d9a8b8e3540 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.694013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b5534-8b40-4904-9767-6d9a8b8e3540 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.694491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b5534-8b40-4904-9767-6d9a8b8e3540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.719770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-008b5534-8b40-4904-9767-6d9a8b8e3540 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:35.720982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-008b5534-8b40-4904-9767-6d9a8b8e3540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:35.722885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51638f68-a5e3-4293-8a97-a0dda2ac0997 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:35.759594 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ac355-bdad-49dd-8e8b-c59bc5a6e7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:35.761408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ac355-bdad-49dd-8e8b-c59bc5a6e7b5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.761690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ac355-bdad-49dd-8e8b-c59bc5a6e7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.761952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ac355-bdad-49dd-8e8b-c59bc5a6e7b5 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.762782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ac355-bdad-49dd-8e8b-c59bc5a6e7b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:35.763074 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ac355-bdad-49dd-8e8b-c59bc5a6e7b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:35.764516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ac355-bdad-49dd-8e8b-c59bc5a6e7b5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:35.781698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 355/712] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:35.781698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ac355-bdad-49dd-8e8b-c59bc5a6e7b5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:35.785200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:35.813294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 358/713] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:35.814478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b5534-8b40-4904-9767-6d9a8b8e3540 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:35.901506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:35.931068 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ac355-bdad-49dd-8e8b-c59bc5a6e7b5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:35.952001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b5534-8b40-4904-9767-6d9a8b8e3540 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:35.976008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f84e14-d146-4f74-9064-1f872efd2218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:35.976710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84e14-d146-4f74-9064-1f872efd2218 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.976991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84e14-d146-4f74-9064-1f872efd2218 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.977266 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f84e14-d146-4f74-9064-1f872efd2218 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.978048 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f84e14-d146-4f74-9064-1f872efd2218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:35.978384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f84e14-d146-4f74-9064-1f872efd2218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:35.980902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f84e14-d146-4f74-9064-1f872efd2218 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:35.994325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8211f-794a-41e1-81b4-c4545276b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:35.995260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8211f-794a-41e1-81b4-c4545276b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:35.995760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8211f-794a-41e1-81b4-c4545276b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:35.995983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 356/714] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:35.996480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8211f-794a-41e1-81b4-c4545276b3e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:35.996827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f84e14-d146-4f74-9064-1f872efd2218 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:35.998036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8211f-794a-41e1-81b4-c4545276b3e0 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=b9129f0b362a4945a0eca7cc33571abd) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:35.998606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8211f-794a-41e1-81b4-c4545276b3e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:36.001307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8211f-794a-41e1-81b4-c4545276b3e0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:36.012514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 359/715] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:35 2022] GET /identity/v3/role_assignments?user.id=b9129f0b362a4945a0eca7cc33571abd&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=04c11dea81274a69974ad5c639f2c60b => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:36.013846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8211f-794a-41e1-81b4-c4545276b3e0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:36.072080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:36.118208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f84e14-d146-4f74-9064-1f872efd2218 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:36.169230 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897bd31-8012-4083-80c6-c089233f5e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:36.170030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897bd31-8012-4083-80c6-c089233f5e64 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:36.170306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897bd31-8012-4083-80c6-c089233f5e64 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:36.170573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897bd31-8012-4083-80c6-c089233f5e64 None admin] PATH_INFO: `/v3/services/network` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:36.171269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897bd31-8012-4083-80c6-c089233f5e64 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:36.176066 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897bd31-8012-4083-80c6-c089233f5e64 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-0897bd31-8012-4083-80c6-c089233f5e64 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:36.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:36.208799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:37:36.209651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:36.213495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8211f-794a-41e1-81b4-c4545276b3e0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:36.215258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 357/716] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:35 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:36.218316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897bd31-8012-4083-80c6-c089233f5e64 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:36.269859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:36.289462 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f3c1c-d7bf-4fd6-8d20-112e309766ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:36.290164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f3c1c-d7bf-4fd6-8d20-112e309766ce None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:36.290448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f3c1c-d7bf-4fd6-8d20-112e309766ce None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:36.290720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f3c1c-d7bf-4fd6-8d20-112e309766ce None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:36.291971 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f3c1c-d7bf-4fd6-8d20-112e309766ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:36.296805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f3c1c-d7bf-4fd6-8d20-112e309766ce None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:36.369374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 360/717] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:37:36 2022] POST /identity/v3/users => generated 254 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:36.392176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897bd31-8012-4083-80c6-c089233f5e64 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:36.429453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:36.444452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d9ce0-0160-44bc-aba5-bf8ea23825cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:36.445145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d9ce0-0160-44bc-aba5-bf8ea23825cf None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:36.445467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d9ce0-0160-44bc-aba5-bf8ea23825cf None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:36.445751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d9ce0-0160-44bc-aba5-bf8ea23825cf None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:36.446467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d9ce0-0160-44bc-aba5-bf8ea23825cf None admin] RBAC: Adding query filter params (name=network) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:36.446770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d9ce0-0160-44bc-aba5-bf8ea23825cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:36.448223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d9ce0-0160-44bc-aba5-bf8ea23825cf None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:36.456854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 358/718] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:37:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:36.462232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f3c1c-d7bf-4fd6-8d20-112e309766ce None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:36.591418 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f3c1c-d7bf-4fd6-8d20-112e309766ce None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:36.631214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5720b1-2d80-408e-8188-96d2de68a0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:36.631893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5720b1-2d80-408e-8188-96d2de68a0da None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:36.632174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5720b1-2d80-408e-8188-96d2de68a0da None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:36.632474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5720b1-2d80-408e-8188-96d2de68a0da None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:36.633193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5720b1-2d80-408e-8188-96d2de68a0da None admin] RBAC: Adding query filter params (type=network) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:36.633500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5720b1-2d80-408e-8188-96d2de68a0da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:36.634889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5720b1-2d80-408e-8188-96d2de68a0da None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:36.642288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 361/719] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:37:36 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:37.487590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f75f73-fd40-4a3e-98d2-4a1bb9c67435 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:37.488137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f75f73-fd40-4a3e-98d2-4a1bb9c67435 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:37.488137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f75f73-fd40-4a3e-98d2-4a1bb9c67435 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:37.489880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 359/720] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:37.495426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fef93-bcb1-42e2-8a81-b903df820918 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:37.495880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fef93-bcb1-42e2-8a81-b903df820918 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:37.496059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fef93-bcb1-42e2-8a81-b903df820918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:37.517851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-756fef93-bcb1-42e2-8a81-b903df820918 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:37.519025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-756fef93-bcb1-42e2-8a81-b903df820918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:37.582122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:37.610091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 362/721] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:37.644583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f75f73-fd40-4a3e-98d2-4a1bb9c67435 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:37.794153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f75f73-fd40-4a3e-98d2-4a1bb9c67435 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:37.835826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e89ab1-ec3f-424a-959e-7843a163907f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:37.836449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e89ab1-ec3f-424a-959e-7843a163907f None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:37.836738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e89ab1-ec3f-424a-959e-7843a163907f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:37.837020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e89ab1-ec3f-424a-959e-7843a163907f None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:37.837767 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e89ab1-ec3f-424a-959e-7843a163907f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:37.839124 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e89ab1-ec3f-424a-959e-7843a163907f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:37.962346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa7a97-1f78-4950-a709-3e2df9e413f2 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:37.962687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa7a97-1f78-4950-a709-3e2df9e413f2 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:37.962970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa7a97-1f78-4950-a709-3e2df9e413f2 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:37.964519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 363/722] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:37:37.965141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 360/723] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:37:37 2022] POST /identity/v3/services => generated 251 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:37.970455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038acec2-8147-44bf-b72a-bc4da786ec9c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:37.971083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038acec2-8147-44bf-b72a-bc4da786ec9c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:37.971579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038acec2-8147-44bf-b72a-bc4da786ec9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:37.997554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-038acec2-8147-44bf-b72a-bc4da786ec9c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:37.999283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-038acec2-8147-44bf-b72a-bc4da786ec9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:38.098488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 364/724] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:38.137926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e89ab1-ec3f-424a-959e-7843a163907f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:38.194259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:38.248248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e89ab1-ec3f-424a-959e-7843a163907f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:38.293055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9c681-3297-470e-9559-2eaef9e877d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:38.293910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9c681-3297-470e-9559-2eaef9e877d3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:38.294201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9c681-3297-470e-9559-2eaef9e877d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:38.294483 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9c681-3297-470e-9559-2eaef9e877d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:38.305888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9c681-3297-470e-9559-2eaef9e877d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-49a9c681-3297-470e-9559-2eaef9e877d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:38.314085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:38.315000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:38.315844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 361/725] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:38.321097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038acec2-8147-44bf-b72a-bc4da786ec9c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:38.431083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038acec2-8147-44bf-b72a-bc4da786ec9c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:38.432688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:38.482073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89114f9-8cd0-4065-a670-9a399ed05f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:38.482750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89114f9-8cd0-4065-a670-9a399ed05f98 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:38.483036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89114f9-8cd0-4065-a670-9a399ed05f98 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:38.483317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89114f9-8cd0-4065-a670-9a399ed05f98 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:38.484674 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89114f9-8cd0-4065-a670-9a399ed05f98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:38.484986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89114f9-8cd0-4065-a670-9a399ed05f98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:38.486356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89114f9-8cd0-4065-a670-9a399ed05f98 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:38.499577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 365/726] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:38.505215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9c681-3297-470e-9559-2eaef9e877d3 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:38.637407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9c681-3297-470e-9559-2eaef9e877d3 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:38.652727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:38.684075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf595c85-38a1-4689-967d-2360da68eebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:38.684832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595c85-38a1-4689-967d-2360da68eebf None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:38.685118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595c85-38a1-4689-967d-2360da68eebf None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:38.685395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595c85-38a1-4689-967d-2360da68eebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:38.700183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf595c85-38a1-4689-967d-2360da68eebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:38.701669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf595c85-38a1-4689-967d-2360da68eebf None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-cf595c85-38a1-4689-967d-2360da68eebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:38.720385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:38.721341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:38.722222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:38.722222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:38.722222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:38.722222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:38.722222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 362/727] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:38.727255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89114f9-8cd0-4065-a670-9a399ed05f98 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:38.870470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89114f9-8cd0-4065-a670-9a399ed05f98 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:38.919843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f36e20-cc5b-4880-888b-cb3efcc32a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:38.920547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f36e20-cc5b-4880-888b-cb3efcc32a55 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:38.920841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f36e20-cc5b-4880-888b-cb3efcc32a55 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:38.921120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f36e20-cc5b-4880-888b-cb3efcc32a55 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:38.922424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f36e20-cc5b-4880-888b-cb3efcc32a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:38.922739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f36e20-cc5b-4880-888b-cb3efcc32a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:38.924114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f36e20-cc5b-4880-888b-cb3efcc32a55 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:38.940862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 366/728] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:38.949693 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf595c85-38a1-4689-967d-2360da68eebf None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:38.984796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d6ba7-85c7-4610-ad22-41d0e3818fcc None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:38.985098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d6ba7-85c7-4610-ad22-41d0e3818fcc None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:38.985658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d6ba7-85c7-4610-ad22-41d0e3818fcc None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:38.987288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 367/729] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:38.993320 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c5724-aee4-46f7-8dfa-774c9b50442e None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:38.993614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c5724-aee4-46f7-8dfa-774c9b50442e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:38.993897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c5724-aee4-46f7-8dfa-774c9b50442e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.025715 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-577c5724-aee4-46f7-8dfa-774c9b50442e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:39.026895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-577c5724-aee4-46f7-8dfa-774c9b50442e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:39.057613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:39.128629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf595c85-38a1-4689-967d-2360da68eebf None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:39.159883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 368/730] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:39.185169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c5724-aee4-46f7-8dfa-774c9b50442e None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:39.201726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bae8f-9540-4c3a-8a22-d46fbbd8c7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:39.202596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bae8f-9540-4c3a-8a22-d46fbbd8c7b6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:39.202883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bae8f-9540-4c3a-8a22-d46fbbd8c7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:39.203161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bae8f-9540-4c3a-8a22-d46fbbd8c7b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.221161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00bae8f-9540-4c3a-8a22-d46fbbd8c7b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:39.222607 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00bae8f-9540-4c3a-8a22-d46fbbd8c7b6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-f00bae8f-9540-4c3a-8a22-d46fbbd8c7b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:39.248271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:39.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:39.250326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 363/731] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:39.256616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bae8f-9540-4c3a-8a22-d46fbbd8c7b6 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:39.296663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c5724-aee4-46f7-8dfa-774c9b50442e None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:39.310773 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:39.344643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fabad-5b83-4d5d-b39c-3ce07c333095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:39.345510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fabad-5b83-4d5d-b39c-3ce07c333095 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:39.345791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fabad-5b83-4d5d-b39c-3ce07c333095 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:39.346062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fabad-5b83-4d5d-b39c-3ce07c333095 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.359560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fabad-5b83-4d5d-b39c-3ce07c333095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-596fabad-5b83-4d5d-b39c-3ce07c333095 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:39.368710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:39.369657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:39.370476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:39.373382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 369/732] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:39.375002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a51e3-1a5c-4ca7-a215-e2f0c03da202 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:39.375508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a51e3-1a5c-4ca7-a215-e2f0c03da202 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:39.375989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a51e3-1a5c-4ca7-a215-e2f0c03da202 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.381606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 370/733] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:39.382721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a51e3-1a5c-4ca7-a215-e2f0c03da202 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:39.401025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:39.408875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bae8f-9540-4c3a-8a22-d46fbbd8c7b6 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:39.450804 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79592e8e-6bfa-497a-87c6-c9c1c46ecb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:39.451861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79592e8e-6bfa-497a-87c6-c9c1c46ecb6b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:39.452328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79592e8e-6bfa-497a-87c6-c9c1c46ecb6b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:39.453335 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79592e8e-6bfa-497a-87c6-c9c1c46ecb6b None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.454390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79592e8e-6bfa-497a-87c6-c9c1c46ecb6b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:39.454893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79592e8e-6bfa-497a-87c6-c9c1c46ecb6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:39.456534 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79592e8e-6bfa-497a-87c6-c9c1c46ecb6b None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:39.475592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 364/734] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 23 15:37:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:39.477158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f690d-35e4-4e76-9023-b5374aa25916 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:39.477664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f690d-35e4-4e76-9023-b5374aa25916 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:39.478120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f690d-35e4-4e76-9023-b5374aa25916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.501349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c65f690d-35e4-4e76-9023-b5374aa25916 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:39.503613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c65f690d-35e4-4e76-9023-b5374aa25916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:39.509199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a51e3-1a5c-4ca7-a215-e2f0c03da202 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:39.550317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c89ca-fc1d-46a4-be19-b6675679f829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:39.550996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c89ca-fc1d-46a4-be19-b6675679f829 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:39.551292 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c89ca-fc1d-46a4-be19-b6675679f829 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:39.551574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c89ca-fc1d-46a4-be19-b6675679f829 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.552331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c89ca-fc1d-46a4-be19-b6675679f829 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:39.552677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c89ca-fc1d-46a4-be19-b6675679f829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:39.554061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c89ca-fc1d-46a4-be19-b6675679f829 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:39.562438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 371/735] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:39.563408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c89ca-fc1d-46a4-be19-b6675679f829 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:39.592380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 365/736] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:39.592380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f09c4-2142-4973-9eef-b88079eda7fd None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:39.592380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f09c4-2142-4973-9eef-b88079eda7fd None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:39.592380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f09c4-2142-4973-9eef-b88079eda7fd None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.592380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 366/737] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:39.592380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f09c4-2142-4973-9eef-b88079eda7fd None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:39.607865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:39.653895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:39.678305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c89ca-fc1d-46a4-be19-b6675679f829 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:39.701020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f09c4-2142-4973-9eef-b88079eda7fd None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:39.715360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372f58a-03ec-4654-a469-461cb4dc8e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:39.715955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372f58a-03ec-4654-a469-461cb4dc8e27 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:39.716241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372f58a-03ec-4654-a469-461cb4dc8e27 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:39.716554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372f58a-03ec-4654-a469-461cb4dc8e27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.734357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd815f5-096e-4eb6-a99c-2872b9cf149a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:39.735163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd815f5-096e-4eb6-a99c-2872b9cf149a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:39.735452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd815f5-096e-4eb6-a99c-2872b9cf149a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:39.735733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd815f5-096e-4eb6-a99c-2872b9cf149a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.746319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd815f5-096e-4eb6-a99c-2872b9cf149a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:39.747749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd815f5-096e-4eb6-a99c-2872b9cf149a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-7bd815f5-096e-4eb6-a99c-2872b9cf149a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:39.766099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:39.766984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:39.767845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:39.767845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:39.767845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:39.767845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:39.767845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 367/738] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:39.768639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92045c64-a0a0-4364-b2a8-28c328f315e8 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:39.768739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92045c64-a0a0-4364-b2a8-28c328f315e8 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:39.769023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92045c64-a0a0-4364-b2a8-28c328f315e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:39.787753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-92045c64-a0a0-4364-b2a8-28c328f315e8 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:39.788963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-92045c64-a0a0-4364-b2a8-28c328f315e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:39.791574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372f58a-03ec-4654-a469-461cb4dc8e27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:39.793069 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372f58a-03ec-4654-a469-461cb4dc8e27 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:39.850506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:39.861235 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 368/739] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 15:37:39.862199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92045c64-a0a0-4364-b2a8-28c328f315e8 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:39.883574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:39.911802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 372/740] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:39 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 15:37:39.912891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372f58a-03ec-4654-a469-461cb4dc8e27 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:39.974033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92045c64-a0a0-4364-b2a8-28c328f315e8 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:40.010354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378e960-22ea-41ae-afeb-1455c21a02e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:40.011285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378e960-22ea-41ae-afeb-1455c21a02e4 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.011565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378e960-22ea-41ae-afeb-1455c21a02e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.011836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378e960-22ea-41ae-afeb-1455c21a02e4 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.012591 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378e960-22ea-41ae-afeb-1455c21a02e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:40.013933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378e960-22ea-41ae-afeb-1455c21a02e4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:40.020170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372f58a-03ec-4654-a469-461cb4dc8e27 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:40.055268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2d2b7-8ead-4048-88c7-9bc8ded206e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:40.055937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2d2b7-8ead-4048-88c7-9bc8ded206e0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.056225 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2d2b7-8ead-4048-88c7-9bc8ded206e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.056530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2d2b7-8ead-4048-88c7-9bc8ded206e0 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.057252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2d2b7-8ead-4048-88c7-9bc8ded206e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:40.057617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2d2b7-8ead-4048-88c7-9bc8ded206e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:40.058981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2d2b7-8ead-4048-88c7-9bc8ded206e0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:40.071204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 373/741] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:40.072018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2d2b7-8ead-4048-88c7-9bc8ded206e0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:40.083299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 369/742] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:37:39 2022] POST /identity/v3/services => generated 240 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:40.084449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378e960-22ea-41ae-afeb-1455c21a02e4 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:40.125239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:40.221401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:40.236812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2d2b7-8ead-4048-88c7-9bc8ded206e0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:40.310516 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a936a44-e759-4e40-a3bf-a98498a02a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:40.311653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a936a44-e759-4e40-a3bf-a98498a02a58 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.312855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a936a44-e759-4e40-a3bf-a98498a02a58 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.313348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a936a44-e759-4e40-a3bf-a98498a02a58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.317034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378e960-22ea-41ae-afeb-1455c21a02e4 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:40.336127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a936a44-e759-4e40-a3bf-a98498a02a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-0a936a44-e759-4e40-a3bf-a98498a02a58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:40.344252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:40.345215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:40.346056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:40.346693 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 374/743] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:40.347699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a936a44-e759-4e40-a3bf-a98498a02a58 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:40.359317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c1f78-da83-49fa-84ca-1ff1b1dbfad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:40.360155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c1f78-da83-49fa-84ca-1ff1b1dbfad1 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.360477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c1f78-da83-49fa-84ca-1ff1b1dbfad1 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.360754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c1f78-da83-49fa-84ca-1ff1b1dbfad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.373981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c1f78-da83-49fa-84ca-1ff1b1dbfad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:40.376041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c1f78-da83-49fa-84ca-1ff1b1dbfad1 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-0f6c1f78-da83-49fa-84ca-1ff1b1dbfad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:40.395866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:40.397072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:40.397955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:40.397955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:40.397955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:40.397955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:40.402005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 370/744] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:40.403041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c1f78-da83-49fa-84ca-1ff1b1dbfad1 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:40.460394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:40.468083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a936a44-e759-4e40-a3bf-a98498a02a58 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:40.506064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce01a11-9529-4601-b3b5-0bc5f8514edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:40.506865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01a11-9529-4601-b3b5-0bc5f8514edb None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.507147 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01a11-9529-4601-b3b5-0bc5f8514edb None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.507421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce01a11-9529-4601-b3b5-0bc5f8514edb None admin] PATH_INFO: `/v3/users/nova` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.519014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce01a11-9529-4601-b3b5-0bc5f8514edb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:40.521212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce01a11-9529-4601-b3b5-0bc5f8514edb None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:40.533366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:40.537761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c1f78-da83-49fa-84ca-1ff1b1dbfad1 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-5ce01a11-9529-4601-b3b5-0bc5f8514edb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:40.541743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:40.544940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:40.546097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 375/745] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 23 15:37:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:40.547939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce01a11-9529-4601-b3b5-0bc5f8514edb None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:40.591542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92d4fb-6d42-4f3b-ac66-380e970e7154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:40.592220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92d4fb-6d42-4f3b-ac66-380e970e7154 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.592539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92d4fb-6d42-4f3b-ac66-380e970e7154 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.593075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92d4fb-6d42-4f3b-ac66-380e970e7154 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.593668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92d4fb-6d42-4f3b-ac66-380e970e7154 None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:40.593899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92d4fb-6d42-4f3b-ac66-380e970e7154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:40.595365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92d4fb-6d42-4f3b-ac66-380e970e7154 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:40.604187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 371/746] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:40.605105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92d4fb-6d42-4f3b-ac66-380e970e7154 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:40.685016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce01a11-9529-4601-b3b5-0bc5f8514edb None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:40.728450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92d4fb-6d42-4f3b-ac66-380e970e7154 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:40.733009 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955ecd3-3d2d-4fc0-91cf-8921e314e13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:40.734501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955ecd3-3d2d-4fc0-91cf-8921e314e13f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.735255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955ecd3-3d2d-4fc0-91cf-8921e314e13f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.735734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955ecd3-3d2d-4fc0-91cf-8921e314e13f None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.740396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955ecd3-3d2d-4fc0-91cf-8921e314e13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:40.740396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955ecd3-3d2d-4fc0-91cf-8921e314e13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:40.740396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955ecd3-3d2d-4fc0-91cf-8921e314e13f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:40.760061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 376/747] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:40.761822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e814622-ef4a-4666-a36b-5a015deb02a1 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.761957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e814622-ef4a-4666-a36b-5a015deb02a1 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.762176 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e814622-ef4a-4666-a36b-5a015deb02a1 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.767877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 377/748] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:40.768681 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e814622-ef4a-4666-a36b-5a015deb02a1 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:40.770985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c702f8e-d4c0-49e3-96b1-0b7a20ce1b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:40.771666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702f8e-d4c0-49e3-96b1-0b7a20ce1b7a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.771953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702f8e-d4c0-49e3-96b1-0b7a20ce1b7a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.772231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702f8e-d4c0-49e3-96b1-0b7a20ce1b7a None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.773350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c702f8e-d4c0-49e3-96b1-0b7a20ce1b7a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:40.773662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c702f8e-d4c0-49e3-96b1-0b7a20ce1b7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:40.775082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c702f8e-d4c0-49e3-96b1-0b7a20ce1b7a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:40.784917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:40.798564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 372/749] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 23 15:37:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:40.798564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c702f8e-d4c0-49e3-96b1-0b7a20ce1b7a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:40.820898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:40.880920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e814622-ef4a-4666-a36b-5a015deb02a1 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:40.904033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c702f8e-d4c0-49e3-96b1-0b7a20ce1b7a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:40.916204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddcce2-38bf-4818-b75d-ad3b81de5d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:40.917023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddcce2-38bf-4818-b75d-ad3b81de5d9d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.917306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddcce2-38bf-4818-b75d-ad3b81de5d9d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.917580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddcce2-38bf-4818-b75d-ad3b81de5d9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.928783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddcce2-38bf-4818-b75d-ad3b81de5d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:40.930360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddcce2-38bf-4818-b75d-ad3b81de5d9d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:40.938554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee9a3d-b2c6-4e66-b557-d9ca57520761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:40.939342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee9a3d-b2c6-4e66-b557-d9ca57520761 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.939622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee9a3d-b2c6-4e66-b557-d9ca57520761 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.939892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee9a3d-b2c6-4e66-b557-d9ca57520761 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.946661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee9a3d-b2c6-4e66-b557-d9ca57520761 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:40.948071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee9a3d-b2c6-4e66-b557-d9ca57520761 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-39ddcce2-38bf-4818-b75d-ad3b81de5d9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:40.948465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:40.949305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:40.950198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:40.950198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:40.950198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:40.950198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:40.950198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 378/750] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:40.950918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafad90-6a01-41be-85e0-68476e817d36 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:40.951078 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafad90-6a01-41be-85e0-68476e817d36 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:40.951357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafad90-6a01-41be-85e0-68476e817d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-29ee9a3d-b2c6-4e66-b557-d9ca57520761 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:40.968919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:40.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:40.975729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:40.975729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:40.975729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:40.975729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:40.975729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:40.975729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:40.975729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:40.975729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:40.976264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-dcafad90-6a01-41be-85e0-68476e817d36 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:40.976350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 373/751] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:40.976459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-dcafad90-6a01-41be-85e0-68476e817d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:40.976541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee9a3d-b2c6-4e66-b557-d9ca57520761 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.037260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:41.055274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 379/752] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:41.055274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafad90-6a01-41be-85e0-68476e817d36 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.064014 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:41.086583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee9a3d-b2c6-4e66-b557-d9ca57520761 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:41.122497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28b20d-a7a6-4cd9-8ce6-c141389234c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:41.123271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28b20d-a7a6-4cd9-8ce6-c141389234c2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:41.123555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28b20d-a7a6-4cd9-8ce6-c141389234c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:41.123830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28b20d-a7a6-4cd9-8ce6-c141389234c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:41.134617 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28b20d-a7a6-4cd9-8ce6-c141389234c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:41.136295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28b20d-a7a6-4cd9-8ce6-c141389234c2 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:41.152516 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafad90-6a01-41be-85e0-68476e817d36 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-9b28b20d-a7a6-4cd9-8ce6-c141389234c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:41.153969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:41.155028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:41.156582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:41.156582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:41.156582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:41.156582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:41.156582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 374/753] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:41.156582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28b20d-a7a6-4cd9-8ce6-c141389234c2 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.216064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d242d6b-8697-4365-8fcd-fd801e43c671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:41.216987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d242d6b-8697-4365-8fcd-fd801e43c671 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:41.217413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d242d6b-8697-4365-8fcd-fd801e43c671 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:41.217820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d242d6b-8697-4365-8fcd-fd801e43c671 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:41.219200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d242d6b-8697-4365-8fcd-fd801e43c671 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:41.219672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d242d6b-8697-4365-8fcd-fd801e43c671 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:41.221198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d242d6b-8697-4365-8fcd-fd801e43c671 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:41.232557 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 380/754] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:41.233621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d242d6b-8697-4365-8fcd-fd801e43c671 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.290145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28b20d-a7a6-4cd9-8ce6-c141389234c2 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:41.302603 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:41.322236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:41.327195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664fac4-7ff4-41b6-8770-88b1e37353b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:41.327959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664fac4-7ff4-41b6-8770-88b1e37353b2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:41.328248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664fac4-7ff4-41b6-8770-88b1e37353b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:41.328551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664fac4-7ff4-41b6-8770-88b1e37353b2 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:41.329340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664fac4-7ff4-41b6-8770-88b1e37353b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:41.329650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664fac4-7ff4-41b6-8770-88b1e37353b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:41.331064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664fac4-7ff4-41b6-8770-88b1e37353b2 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:41.343873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 375/755] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:41.345278 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664fac4-7ff4-41b6-8770-88b1e37353b2 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.355605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d242d6b-8697-4365-8fcd-fd801e43c671 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:41.391841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3708d5-e69c-40ae-b8ee-eb64867247ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:41.392720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3708d5-e69c-40ae-b8ee-eb64867247ae None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:41.393004 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3708d5-e69c-40ae-b8ee-eb64867247ae None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:41.393286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3708d5-e69c-40ae-b8ee-eb64867247ae None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:41.393955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3708d5-e69c-40ae-b8ee-eb64867247ae None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:41.395329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3708d5-e69c-40ae-b8ee-eb64867247ae None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-8d3708d5-e69c-40ae-b8ee-eb64867247ae None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:41.408290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:41.409457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:37:41.410569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:41.414417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 381/756] 10.222.0.23 () {56 vars in 1129 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:41.415260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3708d5-e69c-40ae-b8ee-eb64867247ae None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.462072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664fac4-7ff4-41b6-8770-88b1e37353b2 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:41.500504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046ef64-ee7b-474f-ac61-9417855cea34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:41.501421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046ef64-ee7b-474f-ac61-9417855cea34 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:41.501736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046ef64-ee7b-474f-ac61-9417855cea34 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:41.502433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046ef64-ee7b-474f-ac61-9417855cea34 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:41.503430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046ef64-ee7b-474f-ac61-9417855cea34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:41.503740 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046ef64-ee7b-474f-ac61-9417855cea34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:41.505883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046ef64-ee7b-474f-ac61-9417855cea34 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:41.521056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3708d5-e69c-40ae-b8ee-eb64867247ae None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:41.526013 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 376/757] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:41.526013 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046ef64-ee7b-474f-ac61-9417855cea34 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.539416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:41.556524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb1ff7-f2e5-4067-817a-02621cd7c352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:41.557338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb1ff7-f2e5-4067-817a-02621cd7c352 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:41.557622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb1ff7-f2e5-4067-817a-02621cd7c352 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:41.557896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb1ff7-f2e5-4067-817a-02621cd7c352 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:41.566358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb1ff7-f2e5-4067-817a-02621cd7c352 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:41.567784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb1ff7-f2e5-4067-817a-02621cd7c352 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-f4eb1ff7-f2e5-4067-817a-02621cd7c352 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:41.582887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:41.583836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:41.584794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 382/758] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:41.585371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb1ff7-f2e5-4067-817a-02621cd7c352 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.593385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:41.641398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046ef64-ee7b-474f-ac61-9417855cea34 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:41.678622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2698-7851-4a7f-8599-89e0a0214a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:41.679272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2698-7851-4a7f-8599-89e0a0214a43 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:41.679543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2698-7851-4a7f-8599-89e0a0214a43 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:41.680223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2698-7851-4a7f-8599-89e0a0214a43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:41.681088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d2698-7851-4a7f-8599-89e0a0214a43 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=e57de80064b74ad59d33abb83a5d17f1) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:41.681397 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d2698-7851-4a7f-8599-89e0a0214a43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:41.682813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d2698-7851-4a7f-8599-89e0a0214a43 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:41.689116 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 377/759] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/role_assignments?user.id=e57de80064b74ad59d33abb83a5d17f1&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=04c11dea81274a69974ad5c639f2c60b => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:41.690199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2698-7851-4a7f-8599-89e0a0214a43 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.705276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb1ff7-f2e5-4067-817a-02621cd7c352 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:41.744333 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4001fb9-60c7-4296-ac2a-77505abd4708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:41.745376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4001fb9-60c7-4296-ac2a-77505abd4708 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:41.746012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4001fb9-60c7-4296-ac2a-77505abd4708 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:41.746534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4001fb9-60c7-4296-ac2a-77505abd4708 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:41.747721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4001fb9-60c7-4296-ac2a-77505abd4708 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:41.748222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4001fb9-60c7-4296-ac2a-77505abd4708 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:41.750547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4001fb9-60c7-4296-ac2a-77505abd4708 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:41.756591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 383/760] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:41.758011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4001fb9-60c7-4296-ac2a-77505abd4708 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.809447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2698-7851-4a7f-8599-89e0a0214a43 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:41.820547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:41.830519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:41.850084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe0d79-332f-4994-827d-91d120343f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:41.850896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe0d79-332f-4994-827d-91d120343f45 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:41.851175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe0d79-332f-4994-827d-91d120343f45 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:41.851443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe0d79-332f-4994-827d-91d120343f45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:41.860789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe0d79-332f-4994-827d-91d120343f45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:41.862397 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe0d79-332f-4994-827d-91d120343f45 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-51fe0d79-332f-4994-827d-91d120343f45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:41.878243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:41.879203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:41.880728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:41.880728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:41.880728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:41.880728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:41.880728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:41.880728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:41.880728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:41.880728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:41.880728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 378/761] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:41.880728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe0d79-332f-4994-827d-91d120343f45 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:41.902791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4001fb9-60c7-4296-ac2a-77505abd4708 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:41.955395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb4e0f-5d31-4f93-9b99-29420771fff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:41.956464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb4e0f-5d31-4f93-9b99-29420771fff0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:41.956978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb4e0f-5d31-4f93-9b99-29420771fff0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:41.957478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb4e0f-5d31-4f93-9b99-29420771fff0 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:41.958516 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb4e0f-5d31-4f93-9b99-29420771fff0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:41.959031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb4e0f-5d31-4f93-9b99-29420771fff0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:41.960709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb4e0f-5d31-4f93-9b99-29420771fff0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:41.980309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 384/762] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:41.981549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb4e0f-5d31-4f93-9b99-29420771fff0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:42.013692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe0d79-332f-4994-827d-91d120343f45 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:42.050789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15503d7f-35ee-4383-be81-dad856313901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:42.051872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15503d7f-35ee-4383-be81-dad856313901 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:42.052450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15503d7f-35ee-4383-be81-dad856313901 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:42.052934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15503d7f-35ee-4383-be81-dad856313901 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:42.053896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15503d7f-35ee-4383-be81-dad856313901 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:42.054413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15503d7f-35ee-4383-be81-dad856313901 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:42.055964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15503d7f-35ee-4383-be81-dad856313901 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:42.061899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 379/763] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:42.063607 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15503d7f-35ee-4383-be81-dad856313901 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:42.104657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:42.108691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb4e0f-5d31-4f93-9b99-29420771fff0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:42.111090 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:42.154324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5dd82-4276-4745-8ad2-df01a85afc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:42.154995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5dd82-4276-4745-8ad2-df01a85afc6f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:42.155278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5dd82-4276-4745-8ad2-df01a85afc6f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:42.155552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5dd82-4276-4745-8ad2-df01a85afc6f None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:42.157169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5dd82-4276-4745-8ad2-df01a85afc6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:42.157486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5dd82-4276-4745-8ad2-df01a85afc6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:42.159214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5dd82-4276-4745-8ad2-df01a85afc6f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:42.176354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15503d7f-35ee-4383-be81-dad856313901 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:42.176768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 385/764] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:42.177436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5dd82-4276-4745-8ad2-df01a85afc6f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:42.223579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b5362-efa7-427a-8fac-c4b057fa91f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:42.224232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b5362-efa7-427a-8fac-c4b057fa91f2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:42.224554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b5362-efa7-427a-8fac-c4b057fa91f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:42.224829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b5362-efa7-427a-8fac-c4b057fa91f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:42.306305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5dd82-4276-4745-8ad2-df01a85afc6f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:42.318810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b5362-efa7-427a-8fac-c4b057fa91f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:42.320376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b5362-efa7-427a-8fac-c4b057fa91f2 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:42.346245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f3294-2ddc-4a1a-9100-d54733a0b1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:42.347246 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f3294-2ddc-4a1a-9100-d54733a0b1a0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:42.347735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f3294-2ddc-4a1a-9100-d54733a0b1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:42.348438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f3294-2ddc-4a1a-9100-d54733a0b1a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:42.349442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f3294-2ddc-4a1a-9100-d54733a0b1a0 None admin] RBAC: Adding query filter params (interface=public, service_id=e06e570f5bb34730b93103ea7921afe7, region_id=RegionOne) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:42.349960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f3294-2ddc-4a1a-9100-d54733a0b1a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:42.351535 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f3294-2ddc-4a1a-9100-d54733a0b1a0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:42.360030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 386/765] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 23 15:37:42 2022] GET /identity/v3/endpoints?service_id=e06e570f5bb34730b93103ea7921afe7&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:42.361309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f3294-2ddc-4a1a-9100-d54733a0b1a0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:42.371623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:42.389151 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:42.450635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 380/766] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:37:42.451743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b5362-efa7-427a-8fac-c4b057fa91f2 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:42.473889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f3294-2ddc-4a1a-9100-d54733a0b1a0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:42.510203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb42cd-5be3-492e-8009-279fcb5f5c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:42.511353 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb42cd-5be3-492e-8009-279fcb5f5c37 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:42.511636 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb42cd-5be3-492e-8009-279fcb5f5c37 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:42.511908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb42cd-5be3-492e-8009-279fcb5f5c37 None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/b9129f0b362a4945a0eca7cc33571abd/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:42.513984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb42cd-5be3-492e-8009-279fcb5f5c37 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=b9129f0b362a4945a0eca7cc33571abd, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:42.515949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb42cd-5be3-492e-8009-279fcb5f5c37 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:42.556248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b5362-efa7-427a-8fac-c4b057fa91f2 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:42.586120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 387/767] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:37:42 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/b9129f0b362a4945a0eca7cc33571abd/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:42.587523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e314fe-0c49-4125-a521-27c34ff067ad None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:42.587884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e314fe-0c49-4125-a521-27c34ff067ad None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:42.588222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e314fe-0c49-4125-a521-27c34ff067ad None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:42.590014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 388/768] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:42.590626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e314fe-0c49-4125-a521-27c34ff067ad None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:42.603752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cd646-d649-4385-a9f6-9ca4355bc04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:42.605220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd646-d649-4385-a9f6-9ca4355bc04d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:42.605730 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd646-d649-4385-a9f6-9ca4355bc04d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:42.606270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd646-d649-4385-a9f6-9ca4355bc04d None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:42.607254 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cd646-d649-4385-a9f6-9ca4355bc04d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:42.609295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cd646-d649-4385-a9f6-9ca4355bc04d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:42.617182 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 381/769] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 23 15:37:42 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:42.619174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4096dc-175a-4cf0-b1b6-3ab3e04f252a None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:42.619800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4096dc-175a-4cf0-b1b6-3ab3e04f252a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:42.620402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4096dc-175a-4cf0-b1b6-3ab3e04f252a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:42.627562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:42.643080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ca4096dc-175a-4cf0-b1b6-3ab3e04f252a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:42.645248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ca4096dc-175a-4cf0-b1b6-3ab3e04f252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:42.713154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:42.734830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e314fe-0c49-4125-a521-27c34ff067ad None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:42.781060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c05bd-5e1b-4882-9ef0-3b4b858aafd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:42.782245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c05bd-5e1b-4882-9ef0-3b4b858aafd8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:42.782624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c05bd-5e1b-4882-9ef0-3b4b858aafd8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:42.782972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c05bd-5e1b-4882-9ef0-3b4b858aafd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:42.797007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c05bd-5e1b-4882-9ef0-3b4b858aafd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:42.798562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 382/770] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:42.798868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c05bd-5e1b-4882-9ef0-3b4b858aafd8 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:42.824667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4096dc-175a-4cf0-b1b6-3ab3e04f252a None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-075c05bd-5e1b-4882-9ef0-3b4b858aafd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:42.828788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:42.829820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:42.830781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:42.830781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:42.830781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:42.830781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:42.830781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 389/771] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:42.836709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c05bd-5e1b-4882-9ef0-3b4b858aafd8 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:42.953916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4096dc-175a-4cf0-b1b6-3ab3e04f252a None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:42.954690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c05bd-5e1b-4882-9ef0-3b4b858aafd8 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:42.991854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431e805-f0cc-436a-b2fa-fd34cd6329e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:42.992574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431e805-f0cc-436a-b2fa-fd34cd6329e8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:42.992863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431e805-f0cc-436a-b2fa-fd34cd6329e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:42.993144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431e805-f0cc-436a-b2fa-fd34cd6329e8 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:42.993875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431e805-f0cc-436a-b2fa-fd34cd6329e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:42.994187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431e805-f0cc-436a-b2fa-fd34cd6329e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:42.995564 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431e805-f0cc-436a-b2fa-fd34cd6329e8 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:42.998463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:43.000183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a24c9-d8e8-4506-8e26-1c07120597e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:43.001306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a24c9-d8e8-4506-8e26-1c07120597e3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:43.001819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a24c9-d8e8-4506-8e26-1c07120597e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:43.002317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a24c9-d8e8-4506-8e26-1c07120597e3 None admin] PATH_INFO: `/v3/services/network` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:43.003207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a24c9-d8e8-4506-8e26-1c07120597e3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:43.005708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a24c9-d8e8-4506-8e26-1c07120597e3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:43.010592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 390/772] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:43.016843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-409a24c9-d8e8-4506-8e26-1c07120597e3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:43.025871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:43.027107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:37:43.028125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:43.030749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 383/773] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:42 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:43.032346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431e805-f0cc-436a-b2fa-fd34cd6329e8 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:43.036300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a24c9-d8e8-4506-8e26-1c07120597e3 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:43.194744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431e805-f0cc-436a-b2fa-fd34cd6329e8 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:43.230664 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ce7ff-073e-488a-bc0c-ceef22893545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:43.231696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ce7ff-073e-488a-bc0c-ceef22893545 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:43.232125 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ce7ff-073e-488a-bc0c-ceef22893545 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:43.232975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ce7ff-073e-488a-bc0c-ceef22893545 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:43.238472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a24c9-d8e8-4506-8e26-1c07120597e3 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:43.241165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ce7ff-073e-488a-bc0c-ceef22893545 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:43.242766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ce7ff-073e-488a-bc0c-ceef22893545 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-878ce7ff-073e-488a-bc0c-ceef22893545 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:43.259204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:43.260303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:43.261642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:43.261642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:43.261642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:43.261642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:43.261642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:43.261642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:43.261642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:43.261642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:43.263966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 391/774] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:43.269545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ce7ff-073e-488a-bc0c-ceef22893545 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:43.328834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:43.361742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b549c-1685-4901-af96-774f641190a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:43.362468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b549c-1685-4901-af96-774f641190a1 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:43.362751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b549c-1685-4901-af96-774f641190a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:43.363029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b549c-1685-4901-af96-774f641190a1 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:43.363760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b549c-1685-4901-af96-774f641190a1 None admin] RBAC: Adding query filter params (name=network) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:43.364100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b549c-1685-4901-af96-774f641190a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:43.365569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b549c-1685-4901-af96-774f641190a1 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:43.375893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ce7ff-073e-488a-bc0c-ceef22893545 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:43.382416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 384/775] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:37:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:43.389333 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b549c-1685-4901-af96-774f641190a1 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:43.420106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd501b-18e2-4e45-b823-352a009dbd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:43.420802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd501b-18e2-4e45-b823-352a009dbd56 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:43.421088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd501b-18e2-4e45-b823-352a009dbd56 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:43.421372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd501b-18e2-4e45-b823-352a009dbd56 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:43.422723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd501b-18e2-4e45-b823-352a009dbd56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:43.423038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd501b-18e2-4e45-b823-352a009dbd56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:43.424514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd501b-18e2-4e45-b823-352a009dbd56 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:43.450032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 392/776] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:43.454427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd501b-18e2-4e45-b823-352a009dbd56 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:43.509780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:43.573556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b549c-1685-4901-af96-774f641190a1 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:43.632073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd501b-18e2-4e45-b823-352a009dbd56 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:43.638010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d3041-78d4-47f5-94f3-c2d393ec698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:43.638780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d3041-78d4-47f5-94f3-c2d393ec698e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:43.643947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d3041-78d4-47f5-94f3-c2d393ec698e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:43.644316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d3041-78d4-47f5-94f3-c2d393ec698e None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:43.645144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d3041-78d4-47f5-94f3-c2d393ec698e None admin] RBAC: Adding query filter params (type=network) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:43.645535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d3041-78d4-47f5-94f3-c2d393ec698e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:43.646973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d3041-78d4-47f5-94f3-c2d393ec698e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:43.656989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 385/777] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:37:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:43.660865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d3041-78d4-47f5-94f3-c2d393ec698e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:43.680739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:43.698172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e391de4-0eac-44e7-adfe-c6ddba33ca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:43.698932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e391de4-0eac-44e7-adfe-c6ddba33ca48 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:43.701103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e391de4-0eac-44e7-adfe-c6ddba33ca48 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:43.701103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e391de4-0eac-44e7-adfe-c6ddba33ca48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:43.701103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e391de4-0eac-44e7-adfe-c6ddba33ca48 None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=72e29fe3a2b645248eef1d52217324d9) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:43.701103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e391de4-0eac-44e7-adfe-c6ddba33ca48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:43.702621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e391de4-0eac-44e7-adfe-c6ddba33ca48 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:43.715666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 393/778] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:43 2022] GET /identity/v3/role_assignments?user.id=72e29fe3a2b645248eef1d52217324d9&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:43.800113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d3041-78d4-47f5-94f3-c2d393ec698e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:43.839295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:43.854325 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128be3ca-0f77-4f11-8fe8-c6864b376592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:43.855130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128be3ca-0f77-4f11-8fe8-c6864b376592 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:43.855481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128be3ca-0f77-4f11-8fe8-c6864b376592 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:43.855825 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128be3ca-0f77-4f11-8fe8-c6864b376592 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:43.867348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128be3ca-0f77-4f11-8fe8-c6864b376592 None admin] RBAC: Adding query filter params (interface=public, service_id=f37cc1a3c1f742c5bff2cc46fb3cda1a, region_id=RegionOne) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:43.867348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128be3ca-0f77-4f11-8fe8-c6864b376592 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:43.867348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128be3ca-0f77-4f11-8fe8-c6864b376592 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:43.871095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 386/779] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 23 15:37:43 2022] GET /identity/v3/endpoints?service_id=f37cc1a3c1f742c5bff2cc46fb3cda1a&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:43.876824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e391de4-0eac-44e7-adfe-c6ddba33ca48 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:44.000972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e391de4-0eac-44e7-adfe-c6ddba33ca48 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:44.039755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eed8c6-d71e-417f-9557-a0193b4c0ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:44.040770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed8c6-d71e-417f-9557-a0193b4c0ccd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.041202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed8c6-d71e-417f-9557-a0193b4c0ccd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.041640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed8c6-d71e-417f-9557-a0193b4c0ccd None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.042459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eed8c6-d71e-417f-9557-a0193b4c0ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:44.043968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eed8c6-d71e-417f-9557-a0193b4c0ccd None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:44.050024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 394/780] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 23 15:37:43 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:44.281671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6749-d679-49ee-8d72-72c416fc5782 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.281876 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6749-d679-49ee-8d72-72c416fc5782 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.282137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6749-d679-49ee-8d72-72c416fc5782 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.283876 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 387/781] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:44.289730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8aefe-49af-4240-87f6-dc16b38d060d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.290025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8aefe-49af-4240-87f6-dc16b38d060d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.290307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8aefe-49af-4240-87f6-dc16b38d060d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.318073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0ca8aefe-49af-4240-87f6-dc16b38d060d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:44.319553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0ca8aefe-49af-4240-87f6-dc16b38d060d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:44.368506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:44.404248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 395/782] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:37:44.428120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc6749-d679-49ee-8d72-72c416fc5782 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:44.539557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc6749-d679-49ee-8d72-72c416fc5782 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:44.574648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e66a11-8901-45d6-bf87-e146cf6c692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:44.575508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e66a11-8901-45d6-bf87-e146cf6c692a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.575797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e66a11-8901-45d6-bf87-e146cf6c692a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.576075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e66a11-8901-45d6-bf87-e146cf6c692a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.593237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e66a11-8901-45d6-bf87-e146cf6c692a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-c5e66a11-8901-45d6-bf87-e146cf6c692a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:44.601672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:44.602580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:44.603407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:44.605492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 388/783] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:44.610580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8aefe-49af-4240-87f6-dc16b38d060d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:44.773460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8aefe-49af-4240-87f6-dc16b38d060d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:44.823687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334d073-bb34-4f0d-b7e5-41590f262155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:44.824295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334d073-bb34-4f0d-b7e5-41590f262155 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:44.824618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334d073-bb34-4f0d-b7e5-41590f262155 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:44.824903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334d073-bb34-4f0d-b7e5-41590f262155 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:44.825659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334d073-bb34-4f0d-b7e5-41590f262155 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:44.825969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334d073-bb34-4f0d-b7e5-41590f262155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:44.827353 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334d073-bb34-4f0d-b7e5-41590f262155 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:44.840256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 396/784] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:44.846614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e66a11-8901-45d6-bf87-e146cf6c692a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:44.932834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.005167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e66a11-8901-45d6-bf87-e146cf6c692a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.058403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ccd5c-7b43-4279-a55d-9a6f53e62c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.059567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ccd5c-7b43-4279-a55d-9a6f53e62c2d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.060081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ccd5c-7b43-4279-a55d-9a6f53e62c2d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.060595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ccd5c-7b43-4279-a55d-9a6f53e62c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.078367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ccd5c-7b43-4279-a55d-9a6f53e62c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.080084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ccd5c-7b43-4279-a55d-9a6f53e62c2d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-fd2ccd5c-7b43-4279-a55d-9a6f53e62c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:45.102114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:45.103064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:45.103924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:45.103924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:45.103924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:45.103924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:45.105926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 389/785] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:45.111759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334d073-bb34-4f0d-b7e5-41590f262155 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.177549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.264394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334d073-bb34-4f0d-b7e5-41590f262155 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.307435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859cae5-f393-4494-92f7-c66db919b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.308395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859cae5-f393-4494-92f7-c66db919b49c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.308929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859cae5-f393-4494-92f7-c66db919b49c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.309443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859cae5-f393-4494-92f7-c66db919b49c None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.310395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859cae5-f393-4494-92f7-c66db919b49c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:45.310906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859cae5-f393-4494-92f7-c66db919b49c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.313046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859cae5-f393-4494-92f7-c66db919b49c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.324763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 397/786] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:45.332455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ccd5c-7b43-4279-a55d-9a6f53e62c2d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.412243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcfe9b-3180-4cdd-bbcc-91c484b5b6b9 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.412948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcfe9b-3180-4cdd-bbcc-91c484b5b6b9 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.413440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcfe9b-3180-4cdd-bbcc-91c484b5b6b9 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.415265 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 398/787] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:37:45.420683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa530c33-a07b-4114-adc9-2aeb0a2c3c40 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.421214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa530c33-a07b-4114-adc9-2aeb0a2c3c40 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.422264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa530c33-a07b-4114-adc9-2aeb0a2c3c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.447322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-aa530c33-a07b-4114-adc9-2aeb0a2c3c40 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:45.449186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-aa530c33-a07b-4114-adc9-2aeb0a2c3c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.524414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 399/788] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:37:45.536380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ccd5c-7b43-4279-a55d-9a6f53e62c2d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.537982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.552152 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa530c33-a07b-4114-adc9-2aeb0a2c3c40 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.572725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f380a3-426c-4d58-a409-479770b6a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.573530 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f380a3-426c-4d58-a409-479770b6a706 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.573811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f380a3-426c-4d58-a409-479770b6a706 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.574091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f380a3-426c-4d58-a409-479770b6a706 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.581816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f380a3-426c-4d58-a409-479770b6a706 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.583330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f380a3-426c-4d58-a409-479770b6a706 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-e7f380a3-426c-4d58-a409-479770b6a706 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:45.602612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:45.603641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:45.604643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 390/789] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:45.606236 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.610385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f380a3-426c-4d58-a409-479770b6a706 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.726770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f380a3-426c-4d58-a409-479770b6a706 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.730653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa530c33-a07b-4114-adc9-2aeb0a2c3c40 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.763834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061e483-41f2-4b7b-8833-d442b296739a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.764592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061e483-41f2-4b7b-8833-d442b296739a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.764877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061e483-41f2-4b7b-8833-d442b296739a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.765154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061e483-41f2-4b7b-8833-d442b296739a None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.766163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061e483-41f2-4b7b-8833-d442b296739a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:45.766478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061e483-41f2-4b7b-8833-d442b296739a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.767911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061e483-41f2-4b7b-8833-d442b296739a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.774501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aeb82-b91b-42b7-a881-94ca5f255aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.775420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aeb82-b91b-42b7-a881-94ca5f255aff None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.775703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aeb82-b91b-42b7-a881-94ca5f255aff None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.776236 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aeb82-b91b-42b7-a881-94ca5f255aff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.789275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4aeb82-b91b-42b7-a881-94ca5f255aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:45.790781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 391/790] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:45.792232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4e6b2-4791-4ce3-b92f-3e5c06ccef69 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.792552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4e6b2-4791-4ce3-b92f-3e5c06ccef69 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.792832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4e6b2-4791-4ce3-b92f-3e5c06ccef69 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.794425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 392/791] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:45.795183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4e6b2-4791-4ce3-b92f-3e5c06ccef69 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-4b4aeb82-b91b-42b7-a881-94ca5f255aff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:45.801393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:45.802355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:45.803177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:45.803807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 400/792] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:45.805253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51861b3f-47e8-4060-ad0b-8b10641422df None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.805361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51861b3f-47e8-4060-ad0b-8b10641422df None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.805562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51861b3f-47e8-4060-ad0b-8b10641422df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.824925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.829471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-51861b3f-47e8-4060-ad0b-8b10641422df None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:45.831098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-51861b3f-47e8-4060-ad0b-8b10641422df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:45.911901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4e6b2-4791-4ce3-b92f-3e5c06ccef69 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:45.929736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 401/793] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:45.929736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51861b3f-47e8-4060-ad0b-8b10641422df None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:45.957573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:45.964044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603702e0-5e9e-4988-99aa-8ca79b6a68f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:45.965001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603702e0-5e9e-4988-99aa-8ca79b6a68f0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:45.965339 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603702e0-5e9e-4988-99aa-8ca79b6a68f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:45.965677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603702e0-5e9e-4988-99aa-8ca79b6a68f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:45.976860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603702e0-5e9e-4988-99aa-8ca79b6a68f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:45.978404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603702e0-5e9e-4988-99aa-8ca79b6a68f0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-603702e0-5e9e-4988-99aa-8ca79b6a68f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:45.998738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:45.999842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:46.000963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:46.000963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:46.000963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:46.000963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:46.007241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 393/794] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:46.008887 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603702e0-5e9e-4988-99aa-8ca79b6a68f0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.064782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51861b3f-47e8-4060-ad0b-8b10641422df None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.090927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.110982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75009ee1-7155-4323-b58f-5fc9fc52beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.111688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75009ee1-7155-4323-b58f-5fc9fc52beeb None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.112017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75009ee1-7155-4323-b58f-5fc9fc52beeb None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.112244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75009ee1-7155-4323-b58f-5fc9fc52beeb None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.113142 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75009ee1-7155-4323-b58f-5fc9fc52beeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:46.113461 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75009ee1-7155-4323-b58f-5fc9fc52beeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.114968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75009ee1-7155-4323-b58f-5fc9fc52beeb None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.124967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 402/795] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:46.125905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75009ee1-7155-4323-b58f-5fc9fc52beeb None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.136519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603702e0-5e9e-4988-99aa-8ca79b6a68f0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.183854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60123073-183f-4eee-9cc4-c413da9ff78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.185167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60123073-183f-4eee-9cc4-c413da9ff78c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.185673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60123073-183f-4eee-9cc4-c413da9ff78c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.186167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60123073-183f-4eee-9cc4-c413da9ff78c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.187571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60123073-183f-4eee-9cc4-c413da9ff78c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.189730 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60123073-183f-4eee-9cc4-c413da9ff78c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-60123073-183f-4eee-9cc4-c413da9ff78c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:46.205110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:46.206086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 15:37:46.206904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:46.211421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 394/796] 10.222.0.23 () {56 vars in 1129 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:46.212345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60123073-183f-4eee-9cc4-c413da9ff78c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.223734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.251976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75009ee1-7155-4323-b58f-5fc9fc52beeb None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.293839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efce37-a0df-4cad-b373-e727ce528857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.294916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efce37-a0df-4cad-b373-e727ce528857 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.295304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efce37-a0df-4cad-b373-e727ce528857 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.296299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efce37-a0df-4cad-b373-e727ce528857 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.297222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efce37-a0df-4cad-b373-e727ce528857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:46.297916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efce37-a0df-4cad-b373-e727ce528857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.299513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efce37-a0df-4cad-b373-e727ce528857 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.318159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 403/797] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:46.318159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efce37-a0df-4cad-b373-e727ce528857 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.337654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60123073-183f-4eee-9cc4-c413da9ff78c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.373361 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f9b78-8ce5-4033-b016-0e2829eeea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.374155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f9b78-8ce5-4033-b016-0e2829eeea04 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.374440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f9b78-8ce5-4033-b016-0e2829eeea04 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.374923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f9b78-8ce5-4033-b016-0e2829eeea04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.378100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.387539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f9b78-8ce5-4033-b016-0e2829eeea04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.389681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f9b78-8ce5-4033-b016-0e2829eeea04 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-868f9b78-8ce5-4033-b016-0e2829eeea04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:46.406843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:46.407749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:46.408645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:46.408645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:46.408645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:46.408645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:46.408645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 395/798] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:46.409667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9953c35-ecf1-44a0-af0c-310b2952d606 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.409957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9953c35-ecf1-44a0-af0c-310b2952d606 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.410234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9953c35-ecf1-44a0-af0c-310b2952d606 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.411601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 396/799] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:46.412158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9953c35-ecf1-44a0-af0c-310b2952d606 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.429267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efce37-a0df-4cad-b373-e727ce528857 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.465135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528aea4e-bbd4-4a41-8cd3-b7b4e440c69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.465736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528aea4e-bbd4-4a41-8cd3-b7b4e440c69f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.466024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528aea4e-bbd4-4a41-8cd3-b7b4e440c69f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.466304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528aea4e-bbd4-4a41-8cd3-b7b4e440c69f None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.467011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528aea4e-bbd4-4a41-8cd3-b7b4e440c69f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:46.467321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528aea4e-bbd4-4a41-8cd3-b7b4e440c69f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.468749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528aea4e-bbd4-4a41-8cd3-b7b4e440c69f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.473600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 404/800] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:46.474216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528aea4e-bbd4-4a41-8cd3-b7b4e440c69f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.476914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.524088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9953c35-ecf1-44a0-af0c-310b2952d606 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.565016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98c48f-22b4-4639-bb57-0f8fde727d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.565909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98c48f-22b4-4639-bb57-0f8fde727d28 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.566212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98c48f-22b4-4639-bb57-0f8fde727d28 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.566465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98c48f-22b4-4639-bb57-0f8fde727d28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.573562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98c48f-22b4-4639-bb57-0f8fde727d28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.574989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98c48f-22b4-4639-bb57-0f8fde727d28 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.588475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528aea4e-bbd4-4a41-8cd3-b7b4e440c69f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-7f98c48f-22b4-4639-bb57-0f8fde727d28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:46.590093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:46.590972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:46.591890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:46.591890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:46.591890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:46.591890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:46.591890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:46.591890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:46.591890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:46.591890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:46.591890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 397/801] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:46.593739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9d0b6-e332-48fc-b232-96d78d08f6dd None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.593739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9d0b6-e332-48fc-b232-96d78d08f6dd None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.593739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9d0b6-e332-48fc-b232-96d78d08f6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.609647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-3fa9d0b6-e332-48fc-b232-96d78d08f6dd None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:46.611778 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3fa9d0b6-e332-48fc-b232-96d78d08f6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:46.630756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da86fb8-c323-435d-b82a-8d4375da13ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.631431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da86fb8-c323-435d-b82a-8d4375da13ea None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.631718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da86fb8-c323-435d-b82a-8d4375da13ea None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.631997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da86fb8-c323-435d-b82a-8d4375da13ea None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.632749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da86fb8-c323-435d-b82a-8d4375da13ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:46.633059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da86fb8-c323-435d-b82a-8d4375da13ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.634436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da86fb8-c323-435d-b82a-8d4375da13ea None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.647219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 405/802] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:46.648014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da86fb8-c323-435d-b82a-8d4375da13ea None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.666732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.698975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 398/803] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:46.698975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9d0b6-e332-48fc-b232-96d78d08f6dd None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.724667 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.764051 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da86fb8-c323-435d-b82a-8d4375da13ea None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.797291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9d0b6-e332-48fc-b232-96d78d08f6dd None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.797478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec6992-789a-440c-a34f-c458eb661a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.798099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec6992-789a-440c-a34f-c458eb661a14 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.798382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec6992-789a-440c-a34f-c458eb661a14 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.798660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec6992-789a-440c-a34f-c458eb661a14 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.799364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec6992-789a-440c-a34f-c458eb661a14 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:46.799729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec6992-789a-440c-a34f-c458eb661a14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.801124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec6992-789a-440c-a34f-c458eb661a14 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.805799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 406/804] 10.222.0.23 () {56 vars in 1124 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:46.806560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec6992-789a-440c-a34f-c458eb661a14 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.834994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82712c1-5df9-479c-b211-1e507eee9f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.835661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82712c1-5df9-479c-b211-1e507eee9f43 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.835946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82712c1-5df9-479c-b211-1e507eee9f43 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.836224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82712c1-5df9-479c-b211-1e507eee9f43 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.837043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82712c1-5df9-479c-b211-1e507eee9f43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:46.837354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82712c1-5df9-479c-b211-1e507eee9f43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.838769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82712c1-5df9-479c-b211-1e507eee9f43 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.849917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 399/805] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:46.850701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82712c1-5df9-479c-b211-1e507eee9f43 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.894143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.907272 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec6992-789a-440c-a34f-c458eb661a14 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.932703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:46.941644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9a94a-75e4-44b5-9226-0ea4a256b2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.942436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9a94a-75e4-44b5-9226-0ea4a256b2d6 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.942720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9a94a-75e4-44b5-9226-0ea4a256b2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.946902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9a94a-75e4-44b5-9226-0ea4a256b2d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.950420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9a94a-75e4-44b5-9226-0ea4a256b2d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:46.951866 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9a94a-75e4-44b5-9226-0ea4a256b2d6 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:46.956082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82712c1-5df9-479c-b211-1e507eee9f43 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-64f9a94a-75e4-44b5-9226-0ea4a256b2d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:46.965929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:46.966882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:46.967832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 407/806] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:46.968904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237bae1-47c1-4503-b26c-8ed5d253700f None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.969182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237bae1-47c1-4503-b26c-8ed5d253700f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.969460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237bae1-47c1-4503-b26c-8ed5d253700f None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:46.970880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 408/807] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:46.976414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237bae1-47c1-4503-b26c-8ed5d253700f None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:46.993620 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea421d8-1a0a-4fb8-8b37-fd59a188141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:46.994414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea421d8-1a0a-4fb8-8b37-fd59a188141b None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:46.994698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea421d8-1a0a-4fb8-8b37-fd59a188141b None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:46.994971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea421d8-1a0a-4fb8-8b37-fd59a188141b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.005411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea421d8-1a0a-4fb8-8b37-fd59a188141b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-bea421d8-1a0a-4fb8-8b37-fd59a188141b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:47.013249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:47.014334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:47.015394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:47.019169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 400/808] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:47.019913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea421d8-1a0a-4fb8-8b37-fd59a188141b None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:47.077194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237bae1-47c1-4503-b26c-8ed5d253700f None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:47.110718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61959b0b-3040-4cd0-92f9-b3594f187118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:47.111370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61959b0b-3040-4cd0-92f9-b3594f187118 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.111656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61959b0b-3040-4cd0-92f9-b3594f187118 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.111944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61959b0b-3040-4cd0-92f9-b3594f187118 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.112119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:47.112704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61959b0b-3040-4cd0-92f9-b3594f187118 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.114071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61959b0b-3040-4cd0-92f9-b3594f187118 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.115948 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea421d8-1a0a-4fb8-8b37-fd59a188141b None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:47.149669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250205cf-8840-40f9-8ec2-faf8a7b04585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:47.150554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250205cf-8840-40f9-8ec2-faf8a7b04585 None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.150837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250205cf-8840-40f9-8ec2-faf8a7b04585 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.151111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250205cf-8840-40f9-8ec2-faf8a7b04585 None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/e57de80064b74ad59d33abb83a5d17f1/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.153068 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250205cf-8840-40f9-8ec2-faf8a7b04585 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=e57de80064b74ad59d33abb83a5d17f1, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.154481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250205cf-8840-40f9-8ec2-faf8a7b04585 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.238538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 409/809] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 23 15:37:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:47.239370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61959b0b-3040-4cd0-92f9-b3594f187118 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:47.260962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 401/810] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:37:47 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/e57de80064b74ad59d33abb83a5d17f1/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:47.262796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265d3a-f0c7-4062-8d2d-e0781404925b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.263159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265d3a-f0c7-4062-8d2d-e0781404925b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.263449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265d3a-f0c7-4062-8d2d-e0781404925b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.305434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-02265d3a-f0c7-4062-8d2d-e0781404925b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:47.307796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-02265d3a-f0c7-4062-8d2d-e0781404925b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:47.324685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:47.403045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61959b0b-3040-4cd0-92f9-b3594f187118 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:47.410060 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 402/811] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:47.411400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02265d3a-f0c7-4062-8d2d-e0781404925b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:47.451010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44421201-3508-4391-bbce-b58c7a3f7b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:47.451721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44421201-3508-4391-bbce-b58c7a3f7b19 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.452012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44421201-3508-4391-bbce-b58c7a3f7b19 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.453237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44421201-3508-4391-bbce-b58c7a3f7b19 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.453356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44421201-3508-4391-bbce-b58c7a3f7b19 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:47.453654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44421201-3508-4391-bbce-b58c7a3f7b19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.455123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44421201-3508-4391-bbce-b58c7a3f7b19 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.477022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 410/812] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 23 15:37:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:47.478005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44421201-3508-4391-bbce-b58c7a3f7b19 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:47.536451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:47.551963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02265d3a-f0c7-4062-8d2d-e0781404925b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:47.586988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903640b-1dc4-4766-a6e1-6fa521b72996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:47.587795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903640b-1dc4-4766-a6e1-6fa521b72996 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.588087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903640b-1dc4-4766-a6e1-6fa521b72996 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.588389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903640b-1dc4-4766-a6e1-6fa521b72996 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.589142 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903640b-1dc4-4766-a6e1-6fa521b72996 None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:47.589452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903640b-1dc4-4766-a6e1-6fa521b72996 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.590812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903640b-1dc4-4766-a6e1-6fa521b72996 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.600735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 403/813] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:47.601637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903640b-1dc4-4766-a6e1-6fa521b72996 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:47.602270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:47.610629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44421201-3508-4391-bbce-b58c7a3f7b19 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:47.652841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa80368-a418-457c-8dd6-6326f1327fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:47.653643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa80368-a418-457c-8dd6-6326f1327fe7 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.654385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa80368-a418-457c-8dd6-6326f1327fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.654668 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa80368-a418-457c-8dd6-6326f1327fe7 None admin] PATH_INFO: `/v3/services/network` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.655730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa80368-a418-457c-8dd6-6326f1327fe7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.657981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa80368-a418-457c-8dd6-6326f1327fe7 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-4fa80368-a418-457c-8dd6-6326f1327fe7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:47.674386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:47.675425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 15:37:47.676259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:47.677573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 411/814] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:47 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:47.677666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa80368-a418-457c-8dd6-6326f1327fe7 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:47.724030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903640b-1dc4-4766-a6e1-6fa521b72996 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:47.769172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27005d2b-1a2e-49fd-b06d-dc9ca67985a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:47.770616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27005d2b-1a2e-49fd-b06d-dc9ca67985a1 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.770904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27005d2b-1a2e-49fd-b06d-dc9ca67985a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.771186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27005d2b-1a2e-49fd-b06d-dc9ca67985a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.791373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:47.808232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa80368-a418-457c-8dd6-6326f1327fe7 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:47.847103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede77529-6609-4b93-81a2-8975e4c2d4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:47.847702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77529-6609-4b93-81a2-8975e4c2d4b0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:47.848062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77529-6609-4b93-81a2-8975e4c2d4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:47.848431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77529-6609-4b93-81a2-8975e4c2d4b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:47.860424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede77529-6609-4b93-81a2-8975e4c2d4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.862032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede77529-6609-4b93-81a2-8975e4c2d4b0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.866703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27005d2b-1a2e-49fd-b06d-dc9ca67985a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:47.868352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27005d2b-1a2e-49fd-b06d-dc9ca67985a1 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ede77529-6609-4b93-81a2-8975e4c2d4b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:47.887201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:47.888132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:47.889084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:47.889084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:47.889084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:47.889084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:47.890059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 412/815] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:47.891623 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede77529-6609-4b93-81a2-8975e4c2d4b0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:47.908639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:47.998025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 404/816] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:37:48.000206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27005d2b-1a2e-49fd-b06d-dc9ca67985a1 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.012174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede77529-6609-4b93-81a2-8975e4c2d4b0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.059960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:48.060688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20e305-e650-418c-ae57-2248c1cb3f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.061638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20e305-e650-418c-ae57-2248c1cb3f3c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.061920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20e305-e650-418c-ae57-2248c1cb3f3c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.062725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20e305-e650-418c-ae57-2248c1cb3f3c None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.063447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20e305-e650-418c-ae57-2248c1cb3f3c None admin] RBAC: Adding query filter params (name=network) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:48.063758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20e305-e650-418c-ae57-2248c1cb3f3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.065324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20e305-e650-418c-ae57-2248c1cb3f3c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.071207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 413/817] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:37:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:48.071207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20e305-e650-418c-ae57-2248c1cb3f3c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.129789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27005d2b-1a2e-49fd-b06d-dc9ca67985a1 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.152626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:48.168818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83485fc7-32c9-4929-a2c8-1ddd501cf3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.169791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83485fc7-32c9-4929-a2c8-1ddd501cf3a3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.170274 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83485fc7-32c9-4929-a2c8-1ddd501cf3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.170769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83485fc7-32c9-4929-a2c8-1ddd501cf3a3 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.171722 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83485fc7-32c9-4929-a2c8-1ddd501cf3a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:48.172250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83485fc7-32c9-4929-a2c8-1ddd501cf3a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.173815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83485fc7-32c9-4929-a2c8-1ddd501cf3a3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.176028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20e305-e650-418c-ae57-2248c1cb3f3c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.188113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 405/818] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:48.189308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83485fc7-32c9-4929-a2c8-1ddd501cf3a3 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.212295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7e6d9-c421-4b33-9df3-32cf2a6c4eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.213133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7e6d9-c421-4b33-9df3-32cf2a6c4eaa None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.213420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7e6d9-c421-4b33-9df3-32cf2a6c4eaa None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.213696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7e6d9-c421-4b33-9df3-32cf2a6c4eaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.224235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7e6d9-c421-4b33-9df3-32cf2a6c4eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.225694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7e6d9-c421-4b33-9df3-32cf2a6c4eaa None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-98c7e6d9-c421-4b33-9df3-32cf2a6c4eaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:48.243056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:48.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:48.245467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:48.245467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:48.245467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:48.245467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:48.245467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 414/819] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:48.245467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7e6d9-c421-4b33-9df3-32cf2a6c4eaa None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.315051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83485fc7-32c9-4929-a2c8-1ddd501cf3a3 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.323193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:48.347413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7e6d9-c421-4b33-9df3-32cf2a6c4eaa None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.352477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0627975-66ed-4d65-9368-14803d3919b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.353404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0627975-66ed-4d65-9368-14803d3919b2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.353980 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0627975-66ed-4d65-9368-14803d3919b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.354555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0627975-66ed-4d65-9368-14803d3919b2 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.355511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0627975-66ed-4d65-9368-14803d3919b2 None admin] RBAC: Adding query filter params (type=network) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:48.356042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0627975-66ed-4d65-9368-14803d3919b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.357792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0627975-66ed-4d65-9368-14803d3919b2 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.369406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 406/820] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:37:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:48.370663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0627975-66ed-4d65-9368-14803d3919b2 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.374173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:48.383077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed96e0-7d0c-4970-bc7c-a77ef5ea4a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.383900 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed96e0-7d0c-4970-bc7c-a77ef5ea4a33 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.384217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed96e0-7d0c-4970-bc7c-a77ef5ea4a33 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.384544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed96e0-7d0c-4970-bc7c-a77ef5ea4a33 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.392923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed96e0-7d0c-4970-bc7c-a77ef5ea4a33 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.394385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed96e0-7d0c-4970-bc7c-a77ef5ea4a33 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-f7ed96e0-7d0c-4970-bc7c-a77ef5ea4a33 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:48.409629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:48.410662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:48.411615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:48.411615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:48.411615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:48.411615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:48.411615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:48.411615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:48.411615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:48.411615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:48.411615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:37:48.411615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:48.422408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 415/821] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 23 15:37:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:48.422408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed96e0-7d0c-4970-bc7c-a77ef5ea4a33 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.510603 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0627975-66ed-4d65-9368-14803d3919b2 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.551362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14126f-507d-4ca8-bb92-6b4f762bbf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.552036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14126f-507d-4ca8-bb92-6b4f762bbf44 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.552578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14126f-507d-4ca8-bb92-6b4f762bbf44 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.552698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14126f-507d-4ca8-bb92-6b4f762bbf44 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.553431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14126f-507d-4ca8-bb92-6b4f762bbf44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:48.553648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14126f-507d-4ca8-bb92-6b4f762bbf44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.555017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14126f-507d-4ca8-bb92-6b4f762bbf44 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.556113 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed96e0-7d0c-4970-bc7c-a77ef5ea4a33 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.571902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 407/822] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:48.571902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14126f-507d-4ca8-bb92-6b4f762bbf44 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.598029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c2ec4-92e9-451d-a83b-dafc052d69de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.598985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c2ec4-92e9-451d-a83b-dafc052d69de None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.599455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c2ec4-92e9-451d-a83b-dafc052d69de None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.599905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c2ec4-92e9-451d-a83b-dafc052d69de None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.600534 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:48.601718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c2ec4-92e9-451d-a83b-dafc052d69de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.603293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c2ec4-92e9-451d-a83b-dafc052d69de None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.670884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14126f-507d-4ca8-bb92-6b4f762bbf44 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.699032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 416/823] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 23 15:37:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:48.700681 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c2ec4-92e9-451d-a83b-dafc052d69de None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.720305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e57cc5-bc61-4160-9358-e76c42a02881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.722019 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e57cc5-bc61-4160-9358-e76c42a02881 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.723119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e57cc5-bc61-4160-9358-e76c42a02881 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.723617 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e57cc5-bc61-4160-9358-e76c42a02881 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.726593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e57cc5-bc61-4160-9358-e76c42a02881 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:48.727820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e57cc5-bc61-4160-9358-e76c42a02881 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.732144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e57cc5-bc61-4160-9358-e76c42a02881 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.777480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 408/824] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 23 15:37:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:48.783373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e57cc5-bc61-4160-9358-e76c42a02881 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:48.821033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:48.902930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c2ec4-92e9-451d-a83b-dafc052d69de None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:48.952673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc469a9-e2b2-4d6c-bb6c-5951bcb874f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:48.953567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc469a9-e2b2-4d6c-bb6c-5951bcb874f4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:48.953857 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc469a9-e2b2-4d6c-bb6c-5951bcb874f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:48.954136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc469a9-e2b2-4d6c-bb6c-5951bcb874f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:48.963809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc469a9-e2b2-4d6c-bb6c-5951bcb874f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:48.965609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc469a9-e2b2-4d6c-bb6c-5951bcb874f4 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:48.966934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ffc469a9-e2b2-4d6c-bb6c-5951bcb874f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:48.984898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:48.985997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:48.986879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:48.986879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:48.986879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:48.986879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:48.986879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:48.986879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:48.986879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:48.986879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:48.986879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 417/825] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:48.993660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc469a9-e2b2-4d6c-bb6c-5951bcb874f4 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.001968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e57cc5-bc61-4160-9358-e76c42a02881 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.044715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cbb4c-b9ba-48f0-8bf8-4dec723353ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.045939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cbb4c-b9ba-48f0-8bf8-4dec723353ba None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.046478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cbb4c-b9ba-48f0-8bf8-4dec723353ba None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.046982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cbb4c-b9ba-48f0-8bf8-4dec723353ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.065476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cbb4c-b9ba-48f0-8bf8-4dec723353ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.066922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cbb4c-b9ba-48f0-8bf8-4dec723353ba None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-233cbb4c-b9ba-48f0-8bf8-4dec723353ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:49.097568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:49.098521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:49.099443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:49.099443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:49.099443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:49.099443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:49.102407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 409/826] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:49.109017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cbb4c-b9ba-48f0-8bf8-4dec723353ba None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.134439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc469a9-e2b2-4d6c-bb6c-5951bcb874f4 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.138500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:49.184039 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914eeb1e-423a-44f2-91f6-253727e01298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eeb1e-423a-44f2-91f6-253727e01298 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.185546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eeb1e-423a-44f2-91f6-253727e01298 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.186085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eeb1e-423a-44f2-91f6-253727e01298 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.187727 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914eeb1e-423a-44f2-91f6-253727e01298 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:49.188296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914eeb1e-423a-44f2-91f6-253727e01298 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.190685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914eeb1e-423a-44f2-91f6-253727e01298 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.210856 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 418/827] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:49.213327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914eeb1e-423a-44f2-91f6-253727e01298 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.240494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cbb4c-b9ba-48f0-8bf8-4dec723353ba None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.275485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d6fe8-c350-4f74-ba9d-e62ccab38103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.276153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6fe8-c350-4f74-ba9d-e62ccab38103 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.276634 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6fe8-c350-4f74-ba9d-e62ccab38103 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.276991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6fe8-c350-4f74-ba9d-e62ccab38103 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.277777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d6fe8-c350-4f74-ba9d-e62ccab38103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:49.278178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d6fe8-c350-4f74-ba9d-e62ccab38103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.283269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d6fe8-c350-4f74-ba9d-e62ccab38103 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.300789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:49.305037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 410/828] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:49.306308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d6fe8-c350-4f74-ba9d-e62ccab38103 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.323917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914eeb1e-423a-44f2-91f6-253727e01298 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.360208 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb03cf8-adc9-43a0-b86e-f9ee4a4cab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.361179 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb03cf8-adc9-43a0-b86e-f9ee4a4cab6a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.361606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb03cf8-adc9-43a0-b86e-f9ee4a4cab6a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.362064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb03cf8-adc9-43a0-b86e-f9ee4a4cab6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.363063 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb03cf8-adc9-43a0-b86e-f9ee4a4cab6a None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=b9129f0b362a4945a0eca7cc33571abd) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:49.363553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb03cf8-adc9-43a0-b86e-f9ee4a4cab6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.365227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb03cf8-adc9-43a0-b86e-f9ee4a4cab6a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.380704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 419/829] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:49 2022] GET /identity/v3/role_assignments?user.id=b9129f0b362a4945a0eca7cc33571abd&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=04c11dea81274a69974ad5c639f2c60b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:49.419828 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d6fe8-c350-4f74-ba9d-e62ccab38103 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.459592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a141e3-94a1-4571-8167-90e60d341a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.460460 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a141e3-94a1-4571-8167-90e60d341a32 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.460815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a141e3-94a1-4571-8167-90e60d341a32 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.461161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a141e3-94a1-4571-8167-90e60d341a32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.469065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a141e3-94a1-4571-8167-90e60d341a32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.470537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a141e3-94a1-4571-8167-90e60d341a32 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-c8a141e3-94a1-4571-8167-90e60d341a32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:49.486054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:49.487081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:49.487962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:49.487962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:49.487962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:49.487962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:49.487962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:49.487962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:49.487962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:49.487962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:49.492092 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 411/830] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:49.495779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb03cf8-adc9-43a0-b86e-f9ee4a4cab6a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.529697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:49.586493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c8222-1ada-4315-8e85-685146bee47a None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.586777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c8222-1ada-4315-8e85-685146bee47a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.587062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c8222-1ada-4315-8e85-685146bee47a None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.588739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 412/831] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:37:49.593889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646e82b-5b7d-49e4-a9a0-6fe025e78d74 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.594180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646e82b-5b7d-49e4-a9a0-6fe025e78d74 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.594465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646e82b-5b7d-49e4-a9a0-6fe025e78d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.611474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb03cf8-adc9-43a0-b86e-f9ee4a4cab6a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.612190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c646e82b-5b7d-49e4-a9a0-6fe025e78d74 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:49.613464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c646e82b-5b7d-49e4-a9a0-6fe025e78d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:49.650005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5efd4a-41b0-4923-b362-7fc6f4e819cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.650673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5efd4a-41b0-4923-b362-7fc6f4e819cf None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.650956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5efd4a-41b0-4923-b362-7fc6f4e819cf None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.651240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5efd4a-41b0-4923-b362-7fc6f4e819cf None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.652020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5efd4a-41b0-4923-b362-7fc6f4e819cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:49.652340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5efd4a-41b0-4923-b362-7fc6f4e819cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.653782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5efd4a-41b0-4923-b362-7fc6f4e819cf None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.665371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 420/832] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:49.670242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a197128-b736-4cc8-b07f-83c64f450274 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.670242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a197128-b736-4cc8-b07f-83c64f450274 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.670242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a197128-b736-4cc8-b07f-83c64f450274 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.670242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 421/833] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:49.671135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a197128-b736-4cc8-b07f-83c64f450274 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.680565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:49.700523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 413/834] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:37:49.700523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433792c-8c51-4950-a72f-0ad612f135cb None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.700523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433792c-8c51-4950-a72f-0ad612f135cb None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.700523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433792c-8c51-4950-a72f-0ad612f135cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.722482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2433792c-8c51-4950-a72f-0ad612f135cb None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:49.723964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2433792c-8c51-4950-a72f-0ad612f135cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:49.774646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:49.787496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a197128-b736-4cc8-b07f-83c64f450274 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.808686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 414/835] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:49.810112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433792c-8c51-4950-a72f-0ad612f135cb None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.823108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8a55e-5a0e-4d23-8b08-63f67afa64da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.824099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8a55e-5a0e-4d23-8b08-63f67afa64da None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.824506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8a55e-5a0e-4d23-8b08-63f67afa64da None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.824786 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8a55e-5a0e-4d23-8b08-63f67afa64da None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/72e29fe3a2b645248eef1d52217324d9/roles/527cc946916f4976ac459aae21d85772` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.840420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8a55e-5a0e-4d23-8b08-63f67afa64da None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=72e29fe3a2b645248eef1d52217324d9, role_id=527cc946916f4976ac459aae21d85772)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:49.842102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8a55e-5a0e-4d23-8b08-63f67afa64da None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:49.914578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433792c-8c51-4950-a72f-0ad612f135cb None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:49.923826 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 422/836] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:37:49 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/72e29fe3a2b645248eef1d52217324d9/roles/527cc946916f4976ac459aae21d85772 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:37:49.924988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8a55e-5a0e-4d23-8b08-63f67afa64da None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:49.956811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:49.966224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fa511-2a0a-4ee5-b749-9543f8ea5abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:49.967465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fa511-2a0a-4ee5-b749-9543f8ea5abb None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:49.968049 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fa511-2a0a-4ee5-b749-9543f8ea5abb None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:49.968554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fa511-2a0a-4ee5-b749-9543f8ea5abb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:49.986144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fa511-2a0a-4ee5-b749-9543f8ea5abb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-f19fa511-2a0a-4ee5-b749-9543f8ea5abb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:49.998959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:50.000598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:37:50.001471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:50.004944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 415/837] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:50.010126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fa511-2a0a-4ee5-b749-9543f8ea5abb None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.056344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8a55e-5a0e-4d23-8b08-63f67afa64da None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.100774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc504baa-4688-48b7-be28-75b19da8e5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.101607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc504baa-4688-48b7-be28-75b19da8e5da None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.101922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc504baa-4688-48b7-be28-75b19da8e5da None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.102845 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc504baa-4688-48b7-be28-75b19da8e5da None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.103540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc504baa-4688-48b7-be28-75b19da8e5da None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.105361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc504baa-4688-48b7-be28-75b19da8e5da None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-fc504baa-4688-48b7-be28-75b19da8e5da None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:50.122276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:50.123630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:50.124660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 423/838] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:50.132403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc504baa-4688-48b7-be28-75b19da8e5da None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.135480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:50.153219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fa511-2a0a-4ee5-b749-9543f8ea5abb None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.201565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee984d-d524-4841-a8a2-4fc442c3cff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.202487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee984d-d524-4841-a8a2-4fc442c3cff9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.202785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee984d-d524-4841-a8a2-4fc442c3cff9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.203097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee984d-d524-4841-a8a2-4fc442c3cff9 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.204290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee984d-d524-4841-a8a2-4fc442c3cff9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:50.204670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee984d-d524-4841-a8a2-4fc442c3cff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.206559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee984d-d524-4841-a8a2-4fc442c3cff9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.219281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 416/839] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:37:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:50.225114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee984d-d524-4841-a8a2-4fc442c3cff9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.253840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc504baa-4688-48b7-be28-75b19da8e5da None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.278797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:50.299512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffffdc-e463-48e5-817b-e46b37880e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.300322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffffdc-e463-48e5-817b-e46b37880e38 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.300733 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffffdc-e463-48e5-817b-e46b37880e38 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.301106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffffdc-e463-48e5-817b-e46b37880e38 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.301912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffffdc-e463-48e5-817b-e46b37880e38 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:50.302340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffffdc-e463-48e5-817b-e46b37880e38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.303810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffffdc-e463-48e5-817b-e46b37880e38 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.308994 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 424/840] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:50.316680 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffffdc-e463-48e5-817b-e46b37880e38 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.367895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee984d-d524-4841-a8a2-4fc442c3cff9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.389844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:50.413394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f0feb-930b-4b92-a6e4-92bf507b82fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.414422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f0feb-930b-4b92-a6e4-92bf507b82fd None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.414858 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f0feb-930b-4b92-a6e4-92bf507b82fd None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.415712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f0feb-930b-4b92-a6e4-92bf507b82fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.428874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffffdc-e463-48e5-817b-e46b37880e38 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.431468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f0feb-930b-4b92-a6e4-92bf507b82fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.433732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f0feb-930b-4b92-a6e4-92bf507b82fd None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-e12f0feb-930b-4b92-a6e4-92bf507b82fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:50.461952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:50.463145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:50.464238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:50.464238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:50.464238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:50.464238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:50.466979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 417/841] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:50.472641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af754f5-69dd-400d-b350-c593f9c0a2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.472927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f0feb-930b-4b92-a6e4-92bf507b82fd None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.473349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af754f5-69dd-400d-b350-c593f9c0a2a2 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.473639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af754f5-69dd-400d-b350-c593f9c0a2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.473915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af754f5-69dd-400d-b350-c593f9c0a2a2 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.474760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af754f5-69dd-400d-b350-c593f9c0a2a2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:50.475077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af754f5-69dd-400d-b350-c593f9c0a2a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.476494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af754f5-69dd-400d-b350-c593f9c0a2a2 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.482181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 425/842] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:50 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:50.582118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:50.592825 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f0feb-930b-4b92-a6e4-92bf507b82fd None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.634672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d1180-d0cf-4116-9db6-1363edc46fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.635365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d1180-d0cf-4116-9db6-1363edc46fc8 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.635650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d1180-d0cf-4116-9db6-1363edc46fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.635922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d1180-d0cf-4116-9db6-1363edc46fc8 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.636973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d1180-d0cf-4116-9db6-1363edc46fc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:50.637282 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d1180-d0cf-4116-9db6-1363edc46fc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.638653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d1180-d0cf-4116-9db6-1363edc46fc8 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.652567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 418/843] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:50.658284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af754f5-69dd-400d-b350-c593f9c0a2a2 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.768970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af754f5-69dd-400d-b350-c593f9c0a2a2 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.798145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:50.804189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421a22a-afe8-46b3-a082-fbe3a3126090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.805023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421a22a-afe8-46b3-a082-fbe3a3126090 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.805308 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421a22a-afe8-46b3-a082-fbe3a3126090 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.805584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421a22a-afe8-46b3-a082-fbe3a3126090 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.815051 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421a22a-afe8-46b3-a082-fbe3a3126090 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.816548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421a22a-afe8-46b3-a082-fbe3a3126090 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-f421a22a-afe8-46b3-a082-fbe3a3126090 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:50.831896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:50.833340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:50.834335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 426/844] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:37:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:50.839132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d1180-d0cf-4116-9db6-1363edc46fc8 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:50.950922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d1180-d0cf-4116-9db6-1363edc46fc8 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:50.986204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de64d27-f265-4541-b658-98bfd1ab98c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:50.986925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de64d27-f265-4541-b658-98bfd1ab98c0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:50.987215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de64d27-f265-4541-b658-98bfd1ab98c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:50.987493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de64d27-f265-4541-b658-98bfd1ab98c0 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:50.988328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de64d27-f265-4541-b658-98bfd1ab98c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:50.988669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de64d27-f265-4541-b658-98bfd1ab98c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:50.996536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de64d27-f265-4541-b658-98bfd1ab98c0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:51.020744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 419/845] 10.222.0.23 () {56 vars in 1135 bytes} [Mon May 23 15:37:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:51.025963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421a22a-afe8-46b3-a082-fbe3a3126090 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:51.136151 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421a22a-afe8-46b3-a082-fbe3a3126090 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:51.172492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee49f1-1a24-4977-bc9c-3c99acbe4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:51.173209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee49f1-1a24-4977-bc9c-3c99acbe4174 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:51.181145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee49f1-1a24-4977-bc9c-3c99acbe4174 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:51.181145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee49f1-1a24-4977-bc9c-3c99acbe4174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:51.235060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:51.245798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee49f1-1a24-4977-bc9c-3c99acbe4174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:51.247268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee49f1-1a24-4977-bc9c-3c99acbe4174 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:51.350474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 427/846] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:51 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:37:51.357443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de64d27-f265-4541-b658-98bfd1ab98c0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:51.369166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:51.463195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de64d27-f265-4541-b658-98bfd1ab98c0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:51.499344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4308161-2930-4bfc-9a1a-3e0e65b0d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:51.501204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4308161-2930-4bfc-9a1a-3e0e65b0d466 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:51.501704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4308161-2930-4bfc-9a1a-3e0e65b0d466 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:51.502188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4308161-2930-4bfc-9a1a-3e0e65b0d466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:51.514747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4308161-2930-4bfc-9a1a-3e0e65b0d466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:51.516680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4308161-2930-4bfc-9a1a-3e0e65b0d466 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-e4308161-2930-4bfc-9a1a-3e0e65b0d466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:51.536114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:51.537134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:51.538052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:51.538052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:51.538052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:51.538052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:51.540275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 420/847] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:51.546033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee49f1-1a24-4977-bc9c-3c99acbe4174 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:51.643198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:51.652638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee49f1-1a24-4977-bc9c-3c99acbe4174 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:51.688587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fded1d3-32e9-4393-b823-93aa07c4a60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:51.689322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fded1d3-32e9-4393-b823-93aa07c4a60c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:51.689610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fded1d3-32e9-4393-b823-93aa07c4a60c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:51.689915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fded1d3-32e9-4393-b823-93aa07c4a60c None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:51.690657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fded1d3-32e9-4393-b823-93aa07c4a60c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:51.690972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fded1d3-32e9-4393-b823-93aa07c4a60c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:51.692443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fded1d3-32e9-4393-b823-93aa07c4a60c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:51.694474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd3ea-5167-4812-aeef-d4edcb72d6e3 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:51.694995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd3ea-5167-4812-aeef-d4edcb72d6e3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:51.695476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd3ea-5167-4812-aeef-d4edcb72d6e3 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:51.697309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 421/848] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:51.704781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 428/849] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:51.705644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe114b-8423-4531-b860-c1f1e4c6a5c8 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:51.705936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe114b-8423-4531-b860-c1f1e4c6a5c8 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:51.706216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe114b-8423-4531-b860-c1f1e4c6a5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:51.710368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fded1d3-32e9-4393-b823-93aa07c4a60c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:51.731424 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-7dbe114b-8423-4531-b860-c1f1e4c6a5c8 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:51.733362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7dbe114b-8423-4531-b860-c1f1e4c6a5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:51.804765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:51.816990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 422/850] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:51.832571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fded1d3-32e9-4393-b823-93aa07c4a60c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:51.844409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe114b-8423-4531-b860-c1f1e4c6a5c8 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:51.879183 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b83f87-7c5e-4336-aac8-25fc6f8d840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:51.880596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b83f87-7c5e-4336-aac8-25fc6f8d840b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:51.881396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b83f87-7c5e-4336-aac8-25fc6f8d840b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:51.882130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b83f87-7c5e-4336-aac8-25fc6f8d840b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:51.890604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b83f87-7c5e-4336-aac8-25fc6f8d840b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:51.893045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b83f87-7c5e-4336-aac8-25fc6f8d840b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-c6b83f87-7c5e-4336-aac8-25fc6f8d840b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:51.912317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:51.913312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:51.914198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:51.914198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:51.914198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:51.914198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:51.914198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:51.914198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:51.914198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:51.914198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:51.916922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 429/851] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:51.923242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b83f87-7c5e-4336-aac8-25fc6f8d840b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:51.939198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:51.964299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe114b-8423-4531-b860-c1f1e4c6a5c8 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.003381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec2e7c-ca95-4315-97fa-68d6ff1c91f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.004593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec2e7c-ca95-4315-97fa-68d6ff1c91f5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.005028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec2e7c-ca95-4315-97fa-68d6ff1c91f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.005477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec2e7c-ca95-4315-97fa-68d6ff1c91f5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.006376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec2e7c-ca95-4315-97fa-68d6ff1c91f5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.008077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec2e7c-ca95-4315-97fa-68d6ff1c91f5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-c9ec2e7c-ca95-4315-97fa-68d6ff1c91f5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:52.022899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:52.024457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:37:52.025320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:52.028531 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 423/852] 10.222.0.23 () {56 vars in 1132 bytes} [Mon May 23 15:37:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:52.035023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec2e7c-ca95-4315-97fa-68d6ff1c91f5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.060599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b83f87-7c5e-4336-aac8-25fc6f8d840b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.097657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024acb5-fb5c-42a4-8f71-530e8a12365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.098076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024acb5-fb5c-42a4-8f71-530e8a12365d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.098364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024acb5-fb5c-42a4-8f71-530e8a12365d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.098644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024acb5-fb5c-42a4-8f71-530e8a12365d None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.099495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024acb5-fb5c-42a4-8f71-530e8a12365d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:52.099813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024acb5-fb5c-42a4-8f71-530e8a12365d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.101277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024acb5-fb5c-42a4-8f71-530e8a12365d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.101930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:52.118426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 430/853] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:52.119435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024acb5-fb5c-42a4-8f71-530e8a12365d None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.165574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec2e7c-ca95-4315-97fa-68d6ff1c91f5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.208609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:52.213008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698aa140-9613-46f6-be1d-cd32dcf1b841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.213686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698aa140-9613-46f6-be1d-cd32dcf1b841 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.214024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698aa140-9613-46f6-be1d-cd32dcf1b841 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.214296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698aa140-9613-46f6-be1d-cd32dcf1b841 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.215005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698aa140-9613-46f6-be1d-cd32dcf1b841 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:52.215565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698aa140-9613-46f6-be1d-cd32dcf1b841 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.216681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698aa140-9613-46f6-be1d-cd32dcf1b841 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.222073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 424/854] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:37:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:52.223093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5003df9-232c-47cb-8fec-3671cc098095 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.223383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5003df9-232c-47cb-8fec-3671cc098095 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.223661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5003df9-232c-47cb-8fec-3671cc098095 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.225193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 425/855] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:52.231295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7316b98-8e7c-4acd-8c62-f722361a3688 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.231647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7316b98-8e7c-4acd-8c62-f722361a3688 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.231979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7316b98-8e7c-4acd-8c62-f722361a3688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.246938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024acb5-fb5c-42a4-8f71-530e8a12365d None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.249485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-d7316b98-8e7c-4acd-8c62-f722361a3688 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:52.251293 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d7316b98-8e7c-4acd-8c62-f722361a3688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:52.290501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03ba75-e721-4722-972c-b39b6bf6eda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.291170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03ba75-e721-4722-972c-b39b6bf6eda5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.291453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03ba75-e721-4722-972c-b39b6bf6eda5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.291733 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03ba75-e721-4722-972c-b39b6bf6eda5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.292610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03ba75-e721-4722-972c-b39b6bf6eda5 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=e57de80064b74ad59d33abb83a5d17f1) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:52.292932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03ba75-e721-4722-972c-b39b6bf6eda5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.295367 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03ba75-e721-4722-972c-b39b6bf6eda5 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.312386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 431/856] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:52 2022] GET /identity/v3/role_assignments?user.id=e57de80064b74ad59d33abb83a5d17f1&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=04c11dea81274a69974ad5c639f2c60b => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:52.312386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03ba75-e721-4722-972c-b39b6bf6eda5 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.336583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 426/857] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:52.351511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7316b98-8e7c-4acd-8c62-f722361a3688 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.406838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:52.450554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03ba75-e721-4722-972c-b39b6bf6eda5 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.479919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7316b98-8e7c-4acd-8c62-f722361a3688 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.490478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6778c58-3012-4214-9613-e59195bb5c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.491567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6778c58-3012-4214-9613-e59195bb5c11 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.492580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6778c58-3012-4214-9613-e59195bb5c11 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.493095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6778c58-3012-4214-9613-e59195bb5c11 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.494650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6778c58-3012-4214-9613-e59195bb5c11 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:52.495275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6778c58-3012-4214-9613-e59195bb5c11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.497217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6778c58-3012-4214-9613-e59195bb5c11 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.500393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:52.513050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 432/858] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:37:52 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:52.531551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa39cf7-2f1d-42bc-952a-ec86b4a762c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.532437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa39cf7-2f1d-42bc-952a-ec86b4a762c9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.532720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa39cf7-2f1d-42bc-952a-ec86b4a762c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.533554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa39cf7-2f1d-42bc-952a-ec86b4a762c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.545930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa39cf7-2f1d-42bc-952a-ec86b4a762c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-4fa39cf7-2f1d-42bc-952a-ec86b4a762c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:52.554003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:52.555010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:52.556288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 427/859] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:37:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:52.561651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6778c58-3012-4214-9613-e59195bb5c11 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.679832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6778c58-3012-4214-9613-e59195bb5c11 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.723877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279bfe0-8c2b-4912-81ac-11fe321fbe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.724613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279bfe0-8c2b-4912-81ac-11fe321fbe2d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.724904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279bfe0-8c2b-4912-81ac-11fe321fbe2d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.725187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279bfe0-8c2b-4912-81ac-11fe321fbe2d None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.725945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279bfe0-8c2b-4912-81ac-11fe321fbe2d None admin] RBAC: Adding query filter params (name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:52.726254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279bfe0-8c2b-4912-81ac-11fe321fbe2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.727632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279bfe0-8c2b-4912-81ac-11fe321fbe2d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.737248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 433/860] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:37:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:52.742604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa39cf7-2f1d-42bc-952a-ec86b4a762c9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.850174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa39cf7-2f1d-42bc-952a-ec86b4a762c9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:52.859916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:52.885756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a28788-5c13-491f-a219-c36c6fe181e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:52.886899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a28788-5c13-491f-a219-c36c6fe181e8 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.889358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a28788-5c13-491f-a219-c36c6fe181e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.889358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a28788-5c13-491f-a219-c36c6fe181e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.889518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f5a9-1ba9-48af-b2e4-0f8e52a9c1ba None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.889518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f5a9-1ba9-48af-b2e4-0f8e52a9c1ba None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.889518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f5a9-1ba9-48af-b2e4-0f8e52a9c1ba None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.891097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 434/861] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:52.896231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c37f7d-79a9-4755-b933-c63df27d0c46 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:52.896568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c37f7d-79a9-4755-b933-c63df27d0c46 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:52.896857 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c37f7d-79a9-4755-b933-c63df27d0c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:52.899085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a28788-5c13-491f-a219-c36c6fe181e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:52.900827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a28788-5c13-491f-a219-c36c6fe181e8 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:52.915272 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-05c37f7d-79a9-4755-b933-c63df27d0c46 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:52.916521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-05c37f7d-79a9-4755-b933-c63df27d0c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-11a28788-5c13-491f-a219-c36c6fe181e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:52.923774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:52.924752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:52.925657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:52.925657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:52.925657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:52.925657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:52.929005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 428/862] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:52.933797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a28788-5c13-491f-a219-c36c6fe181e8 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:52.980577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:52.993920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 435/863] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:37:53.022301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c37f7d-79a9-4755-b933-c63df27d0c46 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:53.051788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a28788-5c13-491f-a219-c36c6fe181e8 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:53.092822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b38477-6e8b-461c-bada-8b44c1a3619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:53.093713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b38477-6e8b-461c-bada-8b44c1a3619d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:53.094138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b38477-6e8b-461c-bada-8b44c1a3619d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:53.094776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b38477-6e8b-461c-bada-8b44c1a3619d None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:53.096044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b38477-6e8b-461c-bada-8b44c1a3619d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:53.096488 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b38477-6e8b-461c-bada-8b44c1a3619d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:53.098490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b38477-6e8b-461c-bada-8b44c1a3619d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:53.123120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 429/864] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:53.123120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b38477-6e8b-461c-bada-8b44c1a3619d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:53.150981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c37f7d-79a9-4755-b933-c63df27d0c46 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:53.151598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:53.190031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f056ca7-44f4-4909-86d3-efdf51d843c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:53.191082 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f056ca7-44f4-4909-86d3-efdf51d843c7 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:53.191578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f056ca7-44f4-4909-86d3-efdf51d843c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:53.192050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f056ca7-44f4-4909-86d3-efdf51d843c7 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:53.192980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f056ca7-44f4-4909-86d3-efdf51d843c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:53.194646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f056ca7-44f4-4909-86d3-efdf51d843c7 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:53.244126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b38477-6e8b-461c-bada-8b44c1a3619d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:53.276839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 436/865] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:37:53 2022] POST /identity/v3/services => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:53.304875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b61baa-9b8b-4ae3-8b1f-cde53825958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:53.305725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b61baa-9b8b-4ae3-8b1f-cde53825958d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:53.306077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b61baa-9b8b-4ae3-8b1f-cde53825958d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:53.306356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b61baa-9b8b-4ae3-8b1f-cde53825958d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:53.315821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b61baa-9b8b-4ae3-8b1f-cde53825958d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:53.317354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b61baa-9b8b-4ae3-8b1f-cde53825958d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-a8b61baa-9b8b-4ae3-8b1f-cde53825958d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:53.334904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:53.335899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:53.336859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:53.336859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:53.336859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:53.336859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:53.336859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:53.336859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:53.336859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:53.336859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:53.336859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:37:53.336859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:53.340304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 430/866] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 23 15:37:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:53.342879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f056ca7-44f4-4909-86d3-efdf51d843c7 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:53.401073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:53.456885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f056ca7-44f4-4909-86d3-efdf51d843c7 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:53.495544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fbc47-6dc9-4c03-a293-a34bb418e14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:53.496294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fbc47-6dc9-4c03-a293-a34bb418e14f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:53.496616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fbc47-6dc9-4c03-a293-a34bb418e14f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:53.496949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fbc47-6dc9-4c03-a293-a34bb418e14f None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:53.497811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fbc47-6dc9-4c03-a293-a34bb418e14f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:53.498128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fbc47-6dc9-4c03-a293-a34bb418e14f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:53.499568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fbc47-6dc9-4c03-a293-a34bb418e14f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:53.518674 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 437/867] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 23 15:37:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:53.524400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b61baa-9b8b-4ae3-8b1f-cde53825958d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:53.609202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:53.633524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b61baa-9b8b-4ae3-8b1f-cde53825958d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:53.672622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f5e8c-31d8-4200-9023-86e19406ca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:53.673209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f5e8c-31d8-4200-9023-86e19406ca32 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:53.673491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f5e8c-31d8-4200-9023-86e19406ca32 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:53.673766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f5e8c-31d8-4200-9023-86e19406ca32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:53.746921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f5e8c-31d8-4200-9023-86e19406ca32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:53.748385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f5e8c-31d8-4200-9023-86e19406ca32 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:53.827070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:53.852348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 431/868] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:53 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:37:53.857746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fbc47-6dc9-4c03-a293-a34bb418e14f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:53.962181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fbc47-6dc9-4c03-a293-a34bb418e14f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:53.973318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:53.997507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fd035-00bf-4f97-bf91-d44b23b19216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:53.998321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd035-00bf-4f97-bf91-d44b23b19216 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:53.998608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd035-00bf-4f97-bf91-d44b23b19216 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:53.998890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd035-00bf-4f97-bf91-d44b23b19216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.010223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fd035-00bf-4f97-bf91-d44b23b19216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:54.012270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fd035-00bf-4f97-bf91-d44b23b19216 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-af9fd035-00bf-4f97-bf91-d44b23b19216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:54.030899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:54.031851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:54.033290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:54.033290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:54.033290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:54.033290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:54.033290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 438/869] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:54.039171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f5e8c-31d8-4200-9023-86e19406ca32 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:54.142355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f5e8c-31d8-4200-9023-86e19406ca32 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:54.177484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94274113-36e4-4cb0-879c-efec4b54f739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:54.178162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94274113-36e4-4cb0-879c-efec4b54f739 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.178444 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94274113-36e4-4cb0-879c-efec4b54f739 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.178723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94274113-36e4-4cb0-879c-efec4b54f739 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.179454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94274113-36e4-4cb0-879c-efec4b54f739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:54.179761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94274113-36e4-4cb0-879c-efec4b54f739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:54.181157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94274113-36e4-4cb0-879c-efec4b54f739 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:54.193581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 432/870] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:54.198901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fd035-00bf-4f97-bf91-d44b23b19216 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:54.311548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fd035-00bf-4f97-bf91-d44b23b19216 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:54.350434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf677188-8db3-4dc0-b29d-a96601b32868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:54.351386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf677188-8db3-4dc0-b29d-a96601b32868 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.351770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf677188-8db3-4dc0-b29d-a96601b32868 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.352168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf677188-8db3-4dc0-b29d-a96601b32868 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.360135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf677188-8db3-4dc0-b29d-a96601b32868 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:54.361971 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf677188-8db3-4dc0-b29d-a96601b32868 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-bf677188-8db3-4dc0-b29d-a96601b32868 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:54.376815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:54.377770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:54.378697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:54.378697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:54.378697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:54.378697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:54.378697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:54.378697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:54.378697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:54.378697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:54.378697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 439/871] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:54.384319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94274113-36e4-4cb0-879c-efec4b54f739 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:54.424764 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:54.492050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94274113-36e4-4cb0-879c-efec4b54f739 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:54.526914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7aeab8-304d-40b0-9876-0e5bd64b0468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:54.527936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aeab8-304d-40b0-9876-0e5bd64b0468 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.528448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aeab8-304d-40b0-9876-0e5bd64b0468 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.528926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aeab8-304d-40b0-9876-0e5bd64b0468 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.529940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7aeab8-304d-40b0-9876-0e5bd64b0468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:54.530481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7aeab8-304d-40b0-9876-0e5bd64b0468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:54.532150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7aeab8-304d-40b0-9876-0e5bd64b0468 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:54.546336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 433/872] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:54.553155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf677188-8db3-4dc0-b29d-a96601b32868 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:54.578861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:54.664614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf677188-8db3-4dc0-b29d-a96601b32868 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:54.701518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2b675-7368-41cb-9e06-9dfc0ea9da95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:54.702547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2b675-7368-41cb-9e06-9dfc0ea9da95 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.703061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2b675-7368-41cb-9e06-9dfc0ea9da95 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.703560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2b675-7368-41cb-9e06-9dfc0ea9da95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.704823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2b675-7368-41cb-9e06-9dfc0ea9da95 None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=72e29fe3a2b645248eef1d52217324d9) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:54.705350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2b675-7368-41cb-9e06-9dfc0ea9da95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:54.706980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2b675-7368-41cb-9e06-9dfc0ea9da95 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:54.720518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 440/873] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:54 2022] GET /identity/v3/role_assignments?user.id=72e29fe3a2b645248eef1d52217324d9&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:54.745474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20797b4c-d75b-4fb8-98a7-3cfb460577cb None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.745808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20797b4c-d75b-4fb8-98a7-3cfb460577cb None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.746038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20797b4c-d75b-4fb8-98a7-3cfb460577cb None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.747814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 434/874] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:54.753662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6427a7c-40f9-4aa3-83b1-69bc04923d76 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.754282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6427a7c-40f9-4aa3-83b1-69bc04923d76 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.754779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6427a7c-40f9-4aa3-83b1-69bc04923d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.773123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d6427a7c-40f9-4aa3-83b1-69bc04923d76 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:54.774728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d6427a7c-40f9-4aa3-83b1-69bc04923d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:54.853525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 441/875] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:54 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:54.868140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532039-aebc-4925-a157-b9a11641b590 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.868822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532039-aebc-4925-a157-b9a11641b590 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.869310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532039-aebc-4925-a157-b9a11641b590 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.872455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 435/876] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:54.876722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6427a7c-40f9-4aa3-83b1-69bc04923d76 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:54.879371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51621ac2-ca93-4e2d-b667-951c06678147 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:54.880033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51621ac2-ca93-4e2d-b667-951c06678147 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:54.880590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51621ac2-ca93-4e2d-b667-951c06678147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:54.900894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:54.901508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-51621ac2-ca93-4e2d-b667-951c06678147 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:54.903517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-51621ac2-ca93-4e2d-b667-951c06678147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:54.995797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 436/877] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:54 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:37:55.002042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6427a7c-40f9-4aa3-83b1-69bc04923d76 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:55.021732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51621ac2-ca93-4e2d-b667-951c06678147 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:55.050768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7d4d4-950b-42b1-9b36-1b043ce50f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:55.051589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7d4d4-950b-42b1-9b36-1b043ce50f5a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:55.051877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7d4d4-950b-42b1-9b36-1b043ce50f5a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:55.052157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7d4d4-950b-42b1-9b36-1b043ce50f5a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:55.052802 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:55.052936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7d4d4-950b-42b1-9b36-1b043ce50f5a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:55.054242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7d4d4-950b-42b1-9b36-1b043ce50f5a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-13c7d4d4-950b-42b1-9b36-1b043ce50f5a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:55.066358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:55.070944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:55.071851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 442/878] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:37:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:55.073626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7d4d4-950b-42b1-9b36-1b043ce50f5a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:55.140116 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51621ac2-ca93-4e2d-b667-951c06678147 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:55.163052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:55.176444 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cbf9f-5797-4490-80d9-c978fd53bce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:55.177126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cbf9f-5797-4490-80d9-c978fd53bce4 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:55.177412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cbf9f-5797-4490-80d9-c978fd53bce4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:55.177693 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cbf9f-5797-4490-80d9-c978fd53bce4 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:55.178440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cbf9f-5797-4490-80d9-c978fd53bce4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:55.179810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cbf9f-5797-4490-80d9-c978fd53bce4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:55.187611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7d4d4-950b-42b1-9b36-1b043ce50f5a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:55.223696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f5f39-0c00-45f5-b12d-c05f66ed6d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:55.224409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f5f39-0c00-45f5-b12d-c05f66ed6d86 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:55.224679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f5f39-0c00-45f5-b12d-c05f66ed6d86 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:55.224952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f5f39-0c00-45f5-b12d-c05f66ed6d86 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:55.225657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f5f39-0c00-45f5-b12d-c05f66ed6d86 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:55.226018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f5f39-0c00-45f5-b12d-c05f66ed6d86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:55.227362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f5f39-0c00-45f5-b12d-c05f66ed6d86 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:55.232547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 443/879] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 23 15:37:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:55.237921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f5f39-0c00-45f5-b12d-c05f66ed6d86 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:55.280651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 437/880] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:37:55 2022] POST /identity/v3/services => generated 269 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:37:55.364342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f5f39-0c00-45f5-b12d-c05f66ed6d86 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:55.408146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df69228-92c4-4049-9059-0fbb56da2d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:55.408774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69228-92c4-4049-9059-0fbb56da2d98 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:55.409054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69228-92c4-4049-9059-0fbb56da2d98 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:55.409326 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69228-92c4-4049-9059-0fbb56da2d98 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:55.410038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df69228-92c4-4049-9059-0fbb56da2d98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:55.410342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df69228-92c4-4049-9059-0fbb56da2d98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:55.411792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df69228-92c4-4049-9059-0fbb56da2d98 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:55.417328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 444/881] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 23 15:37:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:55.638164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d93d8b-f307-4f9e-b64d-2f3053191b67 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:55.638551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d93d8b-f307-4f9e-b64d-2f3053191b67 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:55.638751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d93d8b-f307-4f9e-b64d-2f3053191b67 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:55.640228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 438/882] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:55.646375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a2ba9-a0dc-4a08-a66b-c32ae458b974 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:55.646924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a2ba9-a0dc-4a08-a66b-c32ae458b974 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:55.647438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a2ba9-a0dc-4a08-a66b-c32ae458b974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:55.652744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:55.673547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-c22a2ba9-a0dc-4a08-a66b-c32ae458b974 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:55.673547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c22a2ba9-a0dc-4a08-a66b-c32ae458b974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:55.744445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 445/883] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:55.760466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d93d8b-f307-4f9e-b64d-2f3053191b67 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:55.851794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:55.869534 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d93d8b-f307-4f9e-b64d-2f3053191b67 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:55.904518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87da88-64ee-4090-a538-a5e61bed542c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:55.905341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87da88-64ee-4090-a538-a5e61bed542c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:55.905936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87da88-64ee-4090-a538-a5e61bed542c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:55.906241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87da88-64ee-4090-a538-a5e61bed542c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:55.906895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87da88-64ee-4090-a538-a5e61bed542c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:55.908264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87da88-64ee-4090-a538-a5e61bed542c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-0e87da88-64ee-4090-a538-a5e61bed542c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:55.920206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:55.921263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:55.923151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 439/884] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:55.928140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a2ba9-a0dc-4a08-a66b-c32ae458b974 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:56.037832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a2ba9-a0dc-4a08-a66b-c32ae458b974 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:56.070967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:56.076269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d203b-f1b0-4ffa-bfbb-a0ef43a980dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:56.076966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d203b-f1b0-4ffa-bfbb-a0ef43a980dc None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.077313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d203b-f1b0-4ffa-bfbb-a0ef43a980dc None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.077594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d203b-f1b0-4ffa-bfbb-a0ef43a980dc None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.078331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d203b-f1b0-4ffa-bfbb-a0ef43a980dc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:56.078642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d203b-f1b0-4ffa-bfbb-a0ef43a980dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:56.080015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d203b-f1b0-4ffa-bfbb-a0ef43a980dc None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:56.085184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 446/885] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:56.093475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87da88-64ee-4090-a538-a5e61bed542c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:56.198629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87da88-64ee-4090-a538-a5e61bed542c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:56.233538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3fbd0-e27c-4a4a-9101-43318f58f07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:56.234213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3fbd0-e27c-4a4a-9101-43318f58f07f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.235069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3fbd0-e27c-4a4a-9101-43318f58f07f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.235352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3fbd0-e27c-4a4a-9101-43318f58f07f None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.236069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3fbd0-e27c-4a4a-9101-43318f58f07f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:56.236396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3fbd0-e27c-4a4a-9101-43318f58f07f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:56.237758 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3fbd0-e27c-4a4a-9101-43318f58f07f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:56.247655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 440/886] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:56.249143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d203b-f1b0-4ffa-bfbb-a0ef43a980dc None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:56.353374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d203b-f1b0-4ffa-bfbb-a0ef43a980dc None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:56.387887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ccf85-4d2e-4ab4-be3c-9f001a43a797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:56.388591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ccf85-4d2e-4ab4-be3c-9f001a43a797 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.388881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ccf85-4d2e-4ab4-be3c-9f001a43a797 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.389159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ccf85-4d2e-4ab4-be3c-9f001a43a797 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.389943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ccf85-4d2e-4ab4-be3c-9f001a43a797 None admin] RBAC: Adding query filter params (interface=public, service_id=026410a233274582aaae4bd6d1821aea, region_id=RegionOne) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:56.390255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ccf85-4d2e-4ab4-be3c-9f001a43a797 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:56.391644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ccf85-4d2e-4ab4-be3c-9f001a43a797 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:56.401365 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 447/887] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 23 15:37:56 2022] GET /identity/v3/endpoints?service_id=026410a233274582aaae4bd6d1821aea&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:56.402791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3fbd0-e27c-4a4a-9101-43318f58f07f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:56.432720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:56.505650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3fbd0-e27c-4a4a-9101-43318f58f07f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:56.541626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5fa10-41ed-4867-8a88-6b9ae0415f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:56.542383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5fa10-41ed-4867-8a88-6b9ae0415f27 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:56.542668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5fa10-41ed-4867-8a88-6b9ae0415f27 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:56.542946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5fa10-41ed-4867-8a88-6b9ae0415f27 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:56.543700 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5fa10-41ed-4867-8a88-6b9ae0415f27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:56.545140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5fa10-41ed-4867-8a88-6b9ae0415f27 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:56.556537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 441/888] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 23 15:37:56 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:57.081780 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d658b21-2d40-4859-8321-eb9114220fe5 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.082319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d658b21-2d40-4859-8321-eb9114220fe5 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.082319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d658b21-2d40-4859-8321-eb9114220fe5 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.083989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 448/889] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:57.089484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a1bd7-3d2f-4c1a-8c57-fc3557aafd7c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.089777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a1bd7-3d2f-4c1a-8c57-fc3557aafd7c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.090061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a1bd7-3d2f-4c1a-8c57-fc3557aafd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.107314 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-680a1bd7-3d2f-4c1a-8c57-fc3557aafd7c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:57.108548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-680a1bd7-3d2f-4c1a-8c57-fc3557aafd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:57.180093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 442/890] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:57.203875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d658b21-2d40-4859-8321-eb9114220fe5 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:57.268072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:57.311653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d658b21-2d40-4859-8321-eb9114220fe5 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:57.347598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799727-3594-4070-8da1-48ea8f570ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:57.348335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799727-3594-4070-8da1-48ea8f570ca1 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.348657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799727-3594-4070-8da1-48ea8f570ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.348937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799727-3594-4070-8da1-48ea8f570ca1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.360212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15799727-3594-4070-8da1-48ea8f570ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-15799727-3594-4070-8da1-48ea8f570ca1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:37:57.368336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:57.369298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:57.370161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 449/891] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 23 15:37:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:57.376743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a1bd7-3d2f-4c1a-8c57-fc3557aafd7c None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:57.391700 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:57.481897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a1bd7-3d2f-4c1a-8c57-fc3557aafd7c None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:57.521158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0a5f4-81e5-4af3-a06f-1e010f5e8c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:57.521868 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0a5f4-81e5-4af3-a06f-1e010f5e8c10 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.522153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0a5f4-81e5-4af3-a06f-1e010f5e8c10 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.522430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0a5f4-81e5-4af3-a06f-1e010f5e8c10 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.523223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0a5f4-81e5-4af3-a06f-1e010f5e8c10 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:57.523536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0a5f4-81e5-4af3-a06f-1e010f5e8c10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:57.525097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0a5f4-81e5-4af3-a06f-1e010f5e8c10 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:57.535513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 443/892] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 23 15:37:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:57.540889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799727-3594-4070-8da1-48ea8f570ca1 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:57.644250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799727-3594-4070-8da1-48ea8f570ca1 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:57.664614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:57.679362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edaca2c-c8e2-4a7f-bb48-6a99421f3328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:57.680170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaca2c-c8e2-4a7f-bb48-6a99421f3328 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.680489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaca2c-c8e2-4a7f-bb48-6a99421f3328 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.680772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edaca2c-c8e2-4a7f-bb48-6a99421f3328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.692796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edaca2c-c8e2-4a7f-bb48-6a99421f3328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:57.694260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edaca2c-c8e2-4a7f-bb48-6a99421f3328 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:57.703951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a91c6e-32f5-4cb3-be5a-a5d0ae261c9e None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.704597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a91c6e-32f5-4cb3-be5a-a5d0ae261c9e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.705091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a91c6e-32f5-4cb3-be5a-a5d0ae261c9e None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.711030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 444/893] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:37:57.712819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53553e9c-7eca-4014-8450-969c7d024732 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-9edaca2c-c8e2-4a7f-bb48-6a99421f3328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:57.712939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:57.713816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:57.714686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:57.714686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:57.714686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:57.714686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:57.714686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 450/894] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:57.715755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53553e9c-7eca-4014-8450-969c7d024732 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.716113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53553e9c-7eca-4014-8450-969c7d024732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.720220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edaca2c-c8e2-4a7f-bb48-6a99421f3328 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:57.740216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-53553e9c-7eca-4014-8450-969c7d024732 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:57.741918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-53553e9c-7eca-4014-8450-969c7d024732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:57.820246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:57.825936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 445/895] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:57.827229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65875f-0254-45b6-9424-e601d901a040 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.827521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65875f-0254-45b6-9424-e601d901a040 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.831537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65875f-0254-45b6-9424-e601d901a040 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.831537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 446/896] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:57.834372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15368c-bdf4-4d7f-ad2c-f8e0395640c0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.834662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15368c-bdf4-4d7f-ad2c-f8e0395640c0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.834946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15368c-bdf4-4d7f-ad2c-f8e0395640c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.836384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edaca2c-c8e2-4a7f-bb48-6a99421f3328 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:57.852393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6b15368c-bdf4-4d7f-ad2c-f8e0395640c0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:57.853648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6b15368c-bdf4-4d7f-ad2c-f8e0395640c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:57.876984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f51762-df6b-4e12-b520-cc0b5fb48a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:57.877699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f51762-df6b-4e12-b520-cc0b5fb48a0b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:57.877942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f51762-df6b-4e12-b520-cc0b5fb48a0b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:57.878222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f51762-df6b-4e12-b520-cc0b5fb48a0b None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:57.879091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f51762-df6b-4e12-b520-cc0b5fb48a0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:57.879276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f51762-df6b-4e12-b520-cc0b5fb48a0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:57.880652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f51762-df6b-4e12-b520-cc0b5fb48a0b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:57.898578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 451/897] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:57.898578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f51762-df6b-4e12-b520-cc0b5fb48a0b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:57.930834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:57.940387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 447/898] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:37:57.940387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15368c-bdf4-4d7f-ad2c-f8e0395640c0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:58.009742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f51762-df6b-4e12-b520-cc0b5fb48a0b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:58.041461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15368c-bdf4-4d7f-ad2c-f8e0395640c0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:58.045840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac267b-dab9-4887-aba3-b61069a5423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:58.046655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac267b-dab9-4887-aba3-b61069a5423d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.046966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac267b-dab9-4887-aba3-b61069a5423d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.047245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac267b-dab9-4887-aba3-b61069a5423d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.047895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac267b-dab9-4887-aba3-b61069a5423d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:58.049309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac267b-dab9-4887-aba3-b61069a5423d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-adac267b-dab9-4887-aba3-b61069a5423d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:58.061249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:58.062363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:58.063227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 452/899] 10.222.0.23 () {56 vars in 1132 bytes} [Mon May 23 15:37:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:58.064074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac267b-dab9-4887-aba3-b61069a5423d None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:58.078009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:58.085409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6a6d1-8a9b-4d85-9648-af21e6acfd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:58.086716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6a6d1-8a9b-4d85-9648-af21e6acfd81 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.087014 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6a6d1-8a9b-4d85-9648-af21e6acfd81 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.087386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6a6d1-8a9b-4d85-9648-af21e6acfd81 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.097993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6a6d1-8a9b-4d85-9648-af21e6acfd81 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:58.099664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6a6d1-8a9b-4d85-9648-af21e6acfd81 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-1ba6a6d1-8a9b-4d85-9648-af21e6acfd81 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:37:58.116900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:58.120279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 448/900] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 23 15:37:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:58.121546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6a6d1-8a9b-4d85-9648-af21e6acfd81 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:58.167166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:58.172939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac267b-dab9-4887-aba3-b61069a5423d None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:58.208739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab43cd-6107-499d-86b0-150bb1a7161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:58.209550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab43cd-6107-499d-86b0-150bb1a7161e None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.209834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab43cd-6107-499d-86b0-150bb1a7161e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.210105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab43cd-6107-499d-86b0-150bb1a7161e None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.210826 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab43cd-6107-499d-86b0-150bb1a7161e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:58.212297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab43cd-6107-499d-86b0-150bb1a7161e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:58.228065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6a6d1-8a9b-4d85-9648-af21e6acfd81 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:58.260945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6661b3f-2eb8-4ece-a5e2-29f602084bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:58.261608 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6661b3f-2eb8-4ece-a5e2-29f602084bdc None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.262153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6661b3f-2eb8-4ece-a5e2-29f602084bdc None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.262429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6661b3f-2eb8-4ece-a5e2-29f602084bdc None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.263133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6661b3f-2eb8-4ece-a5e2-29f602084bdc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:58.263441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6661b3f-2eb8-4ece-a5e2-29f602084bdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:58.264831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6661b3f-2eb8-4ece-a5e2-29f602084bdc None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:58.269844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 449/901] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:37:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:58.270604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6661b3f-2eb8-4ece-a5e2-29f602084bdc None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:58.300649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 453/902] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:37:58 2022] POST /identity/v3/services => generated 241 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:58.301686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab43cd-6107-499d-86b0-150bb1a7161e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:58.338034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:58.402213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6661b3f-2eb8-4ece-a5e2-29f602084bdc None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:58.461401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee774777-6182-419a-98b8-cadd7e2d4b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:58.462010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee774777-6182-419a-98b8-cadd7e2d4b69 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.462293 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee774777-6182-419a-98b8-cadd7e2d4b69 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.462575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee774777-6182-419a-98b8-cadd7e2d4b69 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.463433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee774777-6182-419a-98b8-cadd7e2d4b69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:58.463772 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee774777-6182-419a-98b8-cadd7e2d4b69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:58.465265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee774777-6182-419a-98b8-cadd7e2d4b69 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:58.475711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab43cd-6107-499d-86b0-150bb1a7161e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:58.489470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 450/903] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 23 15:37:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:58.490559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee774777-6182-419a-98b8-cadd7e2d4b69 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:58.532860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b64d5ea-d828-4bf7-9959-29185c929e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:58.534037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64d5ea-d828-4bf7-9959-29185c929e9f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.534614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64d5ea-d828-4bf7-9959-29185c929e9f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.535391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64d5ea-d828-4bf7-9959-29185c929e9f None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.537046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b64d5ea-d828-4bf7-9959-29185c929e9f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:58.537378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b64d5ea-d828-4bf7-9959-29185c929e9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:58.538769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b64d5ea-d828-4bf7-9959-29185c929e9f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:58.545495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 454/904] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:37:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:58.551156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b64d5ea-d828-4bf7-9959-29185c929e9f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:58.580003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:58.622679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee774777-6182-419a-98b8-cadd7e2d4b69 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:58.641455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:58.671906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b64d5ea-d828-4bf7-9959-29185c929e9f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:58.675925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ad483-bbb1-4ce0-adfd-388fe9e8f7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:58.676615 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ad483-bbb1-4ce0-adfd-388fe9e8f7b6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.677292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ad483-bbb1-4ce0-adfd-388fe9e8f7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.677589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ad483-bbb1-4ce0-adfd-388fe9e8f7b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.711300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112a55f-6075-46a2-b651-9047e7800a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:58.712201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112a55f-6075-46a2-b651-9047e7800a77 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.712572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112a55f-6075-46a2-b651-9047e7800a77 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.712914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112a55f-6075-46a2-b651-9047e7800a77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.713783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112a55f-6075-46a2-b651-9047e7800a77 None admin] RBAC: Adding query filter params (interface=public, service_id=699027f1f22440228ad6e959f88c6031, region_id=RegionOne) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:58.714168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112a55f-6075-46a2-b651-9047e7800a77 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:58.715619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112a55f-6075-46a2-b651-9047e7800a77 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:58.721459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 455/905] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 23 15:37:58 2022] GET /identity/v3/endpoints?service_id=699027f1f22440228ad6e959f88c6031&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:58.727355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112a55f-6075-46a2-b651-9047e7800a77 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:58.766669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ad483-bbb1-4ce0-adfd-388fe9e8f7b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:58.768080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ad483-bbb1-4ce0-adfd-388fe9e8f7b6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:58.839143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:58.854904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112a55f-6075-46a2-b651-9047e7800a77 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:58.877012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 451/906] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:37:58 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:37:58.877012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23131292-4d5b-4cdc-bc98-5a60f10f4ec1 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.877012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23131292-4d5b-4cdc-bc98-5a60f10f4ec1 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.877012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23131292-4d5b-4cdc-bc98-5a60f10f4ec1 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.877012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 452/907] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:37:58.880501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23131292-4d5b-4cdc-bc98-5a60f10f4ec1 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:58.891557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:58.896566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdb2a8-61c7-468f-afc1-75b40a7022cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:58.897239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdb2a8-61c7-468f-afc1-75b40a7022cf None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.897522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdb2a8-61c7-468f-afc1-75b40a7022cf None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.897800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdb2a8-61c7-468f-afc1-75b40a7022cf None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.898480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdb2a8-61c7-468f-afc1-75b40a7022cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:58.899907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdb2a8-61c7-468f-afc1-75b40a7022cf None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:58.916395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 456/908] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 23 15:37:58 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:37:58.916395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cedd9-9939-4e44-8878-8cd59dcb57cf None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:58.916395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cedd9-9939-4e44-8878-8cd59dcb57cf None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:58.916395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cedd9-9939-4e44-8878-8cd59dcb57cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:58.925997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-5c0cedd9-9939-4e44-8878-8cd59dcb57cf None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:37:58.927222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5c0cedd9-9939-4e44-8878-8cd59dcb57cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:37:59.017805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23131292-4d5b-4cdc-bc98-5a60f10f4ec1 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:59.024224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 457/909] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:37:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:37:59.047630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cedd9-9939-4e44-8878-8cd59dcb57cf None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:59.059253 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e6c2b-fdc7-45d3-b6d3-3d9eb18be2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:59.060543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e6c2b-fdc7-45d3-b6d3-3d9eb18be2a9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:59.061057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e6c2b-fdc7-45d3-b6d3-3d9eb18be2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:59.061542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e6c2b-fdc7-45d3-b6d3-3d9eb18be2a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:59.074033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e6c2b-fdc7-45d3-b6d3-3d9eb18be2a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:59.075903 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e6c2b-fdc7-45d3-b6d3-3d9eb18be2a9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-c86e6c2b-fdc7-45d3-b6d3-3d9eb18be2a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:37:59.095921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:59.096895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:59.097800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:37:59.097800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:37:59.097800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:37:59.097800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:59.100112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 453/910] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:59.105974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e6c2b-fdc7-45d3-b6d3-3d9eb18be2a9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:59.150305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:59.181062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:59.182343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cedd9-9939-4e44-8878-8cd59dcb57cf None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:59.222442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65a5a2-55df-4d8c-8055-ecf9fe662a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:59.223347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65a5a2-55df-4d8c-8055-ecf9fe662a8c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:59.223737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65a5a2-55df-4d8c-8055-ecf9fe662a8c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:59.224024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65a5a2-55df-4d8c-8055-ecf9fe662a8c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:59.224722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65a5a2-55df-4d8c-8055-ecf9fe662a8c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:59.226201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65a5a2-55df-4d8c-8055-ecf9fe662a8c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:59.230283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e6c2b-fdc7-45d3-b6d3-3d9eb18be2a9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-fb65a5a2-55df-4d8c-8055-ecf9fe662a8c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:59.238965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:59.239998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:37:59.240989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 458/911] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:37:59.247513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65a5a2-55df-4d8c-8055-ecf9fe662a8c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:59.275497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcb307-424c-4642-a262-532a4bb459be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:59.276386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcb307-424c-4642-a262-532a4bb459be None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:59.276681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcb307-424c-4642-a262-532a4bb459be None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:59.276962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcb307-424c-4642-a262-532a4bb459be None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:59.277866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcb307-424c-4642-a262-532a4bb459be None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:59.278177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcb307-424c-4642-a262-532a4bb459be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:59.282087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcb307-424c-4642-a262-532a4bb459be None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:59.308618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 454/912] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:59.310073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcb307-424c-4642-a262-532a4bb459be None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:59.363824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65a5a2-55df-4d8c-8055-ecf9fe662a8c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:59.402287 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe82c7c-838f-4ebc-90ca-8427c6b8b597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:59.403230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe82c7c-838f-4ebc-90ca-8427c6b8b597 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:59.403627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe82c7c-838f-4ebc-90ca-8427c6b8b597 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:59.404003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe82c7c-838f-4ebc-90ca-8427c6b8b597 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:59.404852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe82c7c-838f-4ebc-90ca-8427c6b8b597 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:59.405263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe82c7c-838f-4ebc-90ca-8427c6b8b597 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:59.407522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe82c7c-838f-4ebc-90ca-8427c6b8b597 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:59.413341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcb307-424c-4642-a262-532a4bb459be None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:59.417661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 459/913] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:59.418599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe82c7c-838f-4ebc-90ca-8427c6b8b597 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:59.428765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:59.437131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:59.452799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ef3b-f8c0-430e-ad85-5beec867aa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:59.454544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ef3b-f8c0-430e-ad85-5beec867aa6f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:59.455065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ef3b-f8c0-430e-ad85-5beec867aa6f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:59.455557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ef3b-f8c0-430e-ad85-5beec867aa6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:59.464880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624ef3b-f8c0-430e-ad85-5beec867aa6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:59.466744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624ef3b-f8c0-430e-ad85-5beec867aa6f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-3624ef3b-f8c0-430e-ad85-5beec867aa6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:37:59.487301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:37:59.488280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:37:59.489232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:37:59.489232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:37:59.489232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:37:59.489232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:37:59.489232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:37:59.489232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:37:59.489232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:37:59.489232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:37:59.495103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 455/914] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:37:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:37:59.497931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ef3b-f8c0-430e-ad85-5beec867aa6f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:59.540526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe82c7c-838f-4ebc-90ca-8427c6b8b597 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:59.575872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e814a-dc07-43e9-ac58-d3633e6ea9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:59.576636 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e814a-dc07-43e9-ac58-d3633e6ea9bf None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:59.576928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e814a-dc07-43e9-ac58-d3633e6ea9bf None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:59.577204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e814a-dc07-43e9-ac58-d3633e6ea9bf None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:59.577909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e814a-dc07-43e9-ac58-d3633e6ea9bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:59.578750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e814a-dc07-43e9-ac58-d3633e6ea9bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:59.580119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e814a-dc07-43e9-ac58-d3633e6ea9bf None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:59.589922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 460/915] 10.222.0.23 () {56 vars in 1114 bytes} [Mon May 23 15:37:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:37:59.590748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e814a-dc07-43e9-ac58-d3633e6ea9bf None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:59.632720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ef3b-f8c0-430e-ad85-5beec867aa6f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:59.663070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:59.671685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccd947-6a62-4cdd-ab05-0db2e1ce5240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:59.672387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccd947-6a62-4cdd-ab05-0db2e1ce5240 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:59.672707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccd947-6a62-4cdd-ab05-0db2e1ce5240 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:59.687734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccd947-6a62-4cdd-ab05-0db2e1ce5240 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:59.687734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccd947-6a62-4cdd-ab05-0db2e1ce5240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:59.687734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccd947-6a62-4cdd-ab05-0db2e1ce5240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:59.687734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccd947-6a62-4cdd-ab05-0db2e1ce5240 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:59.688273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 456/916] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:37:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:37:59.696576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e814a-dc07-43e9-ac58-d3633e6ea9bf None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:59.697625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccd947-6a62-4cdd-ab05-0db2e1ce5240 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:37:59.738018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da56049-cc94-4efc-9503-a9db8ee7220d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:59.738683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da56049-cc94-4efc-9503-a9db8ee7220d None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:59.738966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da56049-cc94-4efc-9503-a9db8ee7220d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:59.739239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da56049-cc94-4efc-9503-a9db8ee7220d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:59.739974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da56049-cc94-4efc-9503-a9db8ee7220d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:59.741435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da56049-cc94-4efc-9503-a9db8ee7220d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:59.758630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:37:59.817306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccd947-6a62-4cdd-ab05-0db2e1ce5240 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:37:59.864387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 461/917] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 23 15:37:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:37:59.866549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38f973-4974-4da6-8de0-cf46f8261426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:37:59.867241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38f973-4974-4da6-8de0-cf46f8261426 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:37:59.867590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38f973-4974-4da6-8de0-cf46f8261426 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:37:59.867857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38f973-4974-4da6-8de0-cf46f8261426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:37:59.868793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38f973-4974-4da6-8de0-cf46f8261426 None admin] RBAC: Adding query filter params (role.id=8ad5ca9a5ad04e7fbb93d1330cf468e2, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=72e29fe3a2b645248eef1d52217324d9) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:37:59.869114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38f973-4974-4da6-8de0-cf46f8261426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:37:59.870648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38f973-4974-4da6-8de0-cf46f8261426 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:37:59.878717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 457/918] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:37:59 2022] GET /identity/v3/role_assignments?user.id=72e29fe3a2b645248eef1d52217324d9&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=8ad5ca9a5ad04e7fbb93d1330cf468e2 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:00.598525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a7eed-5fa7-45f1-a7c7-6094029f1560 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:00.599030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a7eed-5fa7-45f1-a7c7-6094029f1560 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:00.599030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a7eed-5fa7-45f1-a7c7-6094029f1560 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:00.600607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 462/919] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:00.606061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe17fe7-248a-4b92-87f0-8d6a7a73afd5 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:00.606495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe17fe7-248a-4b92-87f0-8d6a7a73afd5 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:00.606668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe17fe7-248a-4b92-87f0-8d6a7a73afd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:00.623764 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-cbe17fe7-248a-4b92-87f0-8d6a7a73afd5 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:00.624979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-cbe17fe7-248a-4b92-87f0-8d6a7a73afd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:00.702805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 458/920] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:00.725612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a7eed-5fa7-45f1-a7c7-6094029f1560 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:00.837759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a7eed-5fa7-45f1-a7c7-6094029f1560 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:00.864696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:00.873714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd9b72-bdb1-4f3c-9b6d-8674df824917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:00.874748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd9b72-bdb1-4f3c-9b6d-8674df824917 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:00.875799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd9b72-bdb1-4f3c-9b6d-8674df824917 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:00.876293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd9b72-bdb1-4f3c-9b6d-8674df824917 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:00.877187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd9b72-bdb1-4f3c-9b6d-8674df824917 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:00.878752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd9b72-bdb1-4f3c-9b6d-8674df824917 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-35bd9b72-bdb1-4f3c-9b6d-8674df824917 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:00.892041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:00.893077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:00.893933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:00.896012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 463/921] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:38:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:00.901826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe17fe7-248a-4b92-87f0-8d6a7a73afd5 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:00.932276 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:01.005178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe17fe7-248a-4b92-87f0-8d6a7a73afd5 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:01.054602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543cf6b-48ba-4465-8158-4603072d5831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:01.055298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543cf6b-48ba-4465-8158-4603072d5831 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.055589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543cf6b-48ba-4465-8158-4603072d5831 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.055872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543cf6b-48ba-4465-8158-4603072d5831 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.056620 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543cf6b-48ba-4465-8158-4603072d5831 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:01.056937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543cf6b-48ba-4465-8158-4603072d5831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:01.058320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543cf6b-48ba-4465-8158-4603072d5831 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:01.068709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 459/922] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 23 15:38:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:01.070157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd9b72-bdb1-4f3c-9b6d-8674df824917 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:01.177421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd9b72-bdb1-4f3c-9b6d-8674df824917 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:01.208959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed59b1e-4b24-4f80-945f-3b2b09f2dd63 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.209273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed59b1e-4b24-4f80-945f-3b2b09f2dd63 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.209531 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed59b1e-4b24-4f80-945f-3b2b09f2dd63 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.211172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 460/923] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:01.213048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d385701-1c6b-4941-82e7-8cc83f5a1bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:01.213724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d385701-1c6b-4941-82e7-8cc83f5a1bf7 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.214016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d385701-1c6b-4941-82e7-8cc83f5a1bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.214301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d385701-1c6b-4941-82e7-8cc83f5a1bf7 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.215222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d385701-1c6b-4941-82e7-8cc83f5a1bf7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:01.215588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d385701-1c6b-4941-82e7-8cc83f5a1bf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:01.215981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790df3ce-31f5-484b-b97e-894ca83a0d8b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.216274 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790df3ce-31f5-484b-b97e-894ca83a0d8b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.216586 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790df3ce-31f5-484b-b97e-894ca83a0d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.216997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d385701-1c6b-4941-82e7-8cc83f5a1bf7 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:01.227668 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 464/924] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 23 15:38:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:01.230362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d385701-1c6b-4941-82e7-8cc83f5a1bf7 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:01.233249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-790df3ce-31f5-484b-b97e-894ca83a0d8b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:01.234440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-790df3ce-31f5-484b-b97e-894ca83a0d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:01.305991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:01.330677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 461/925] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:01.355482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790df3ce-31f5-484b-b97e-894ca83a0d8b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:01.364913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d385701-1c6b-4941-82e7-8cc83f5a1bf7 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:01.383354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:01.400042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b67643-14a3-4a79-b0ad-9b92d00520aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:01.400742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b67643-14a3-4a79-b0ad-9b92d00520aa None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.401025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b67643-14a3-4a79-b0ad-9b92d00520aa None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.401472 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b67643-14a3-4a79-b0ad-9b92d00520aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.402412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b67643-14a3-4a79-b0ad-9b92d00520aa None admin] RBAC: Adding query filter params (interface=public, service_id=1548b5d72c6c4d23a5938af8a417d591, region_id=RegionOne) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:01.402929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b67643-14a3-4a79-b0ad-9b92d00520aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:01.404486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b67643-14a3-4a79-b0ad-9b92d00520aa None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:01.410931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 465/926] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 23 15:38:01 2022] GET /identity/v3/endpoints?service_id=1548b5d72c6c4d23a5938af8a417d591&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:01.417797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b67643-14a3-4a79-b0ad-9b92d00520aa None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:01.484429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790df3ce-31f5-484b-b97e-894ca83a0d8b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:01.532023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff0bdc-2b21-41e2-a37c-d5e8d5540827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:01.533161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff0bdc-2b21-41e2-a37c-d5e8d5540827 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.533597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff0bdc-2b21-41e2-a37c-d5e8d5540827 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.534021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff0bdc-2b21-41e2-a37c-d5e8d5540827 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.535356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff0bdc-2b21-41e2-a37c-d5e8d5540827 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:01.537190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff0bdc-2b21-41e2-a37c-d5e8d5540827 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:01.548814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b67643-14a3-4a79-b0ad-9b92d00520aa None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-97ff0bdc-2b21-41e2-a37c-d5e8d5540827 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:01.554613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:01.555765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 15:38:01.556851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:01.559968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 462/927] 10.222.0.23 () {56 vars in 1132 bytes} [Mon May 23 15:38:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:01.566445 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff0bdc-2b21-41e2-a37c-d5e8d5540827 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:01.575942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:01.594412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a18c6-3843-482f-93bb-1f4be6e9d485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:01.599243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a18c6-3843-482f-93bb-1f4be6e9d485 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.599243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a18c6-3843-482f-93bb-1f4be6e9d485 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.599243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a18c6-3843-482f-93bb-1f4be6e9d485 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.599243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a18c6-3843-482f-93bb-1f4be6e9d485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:01.599243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a18c6-3843-482f-93bb-1f4be6e9d485 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:01.609114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 466/928] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 23 15:38:01 2022] GET /identity/v3/services => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:38:01.676188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:01.695676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff0bdc-2b21-41e2-a37c-d5e8d5540827 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:01.736398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adad5f-f737-4860-9681-efe954e198ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:01.737340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adad5f-f737-4860-9681-efe954e198ed None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.737795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adad5f-f737-4860-9681-efe954e198ed None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.738231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adad5f-f737-4860-9681-efe954e198ed None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.739115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adad5f-f737-4860-9681-efe954e198ed None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:01.739580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adad5f-f737-4860-9681-efe954e198ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:01.741151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adad5f-f737-4860-9681-efe954e198ed None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:01.752294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 463/929] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:38:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:01.753961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a18c6-3843-482f-93bb-1f4be6e9d485 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:01.864910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a18c6-3843-482f-93bb-1f4be6e9d485 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:01.900450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b59bc-d86c-4a78-b717-0106cd21c64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:01.901402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b59bc-d86c-4a78-b717-0106cd21c64d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:01.901858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b59bc-d86c-4a78-b717-0106cd21c64d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:01.902291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b59bc-d86c-4a78-b717-0106cd21c64d None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:01.903145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b59bc-d86c-4a78-b717-0106cd21c64d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:01.903601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b59bc-d86c-4a78-b717-0106cd21c64d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:01.905149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b59bc-d86c-4a78-b717-0106cd21c64d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:01.915020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 467/930] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:38:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:01.916575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adad5f-f737-4860-9681-efe954e198ed None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.031621 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adad5f-f737-4860-9681-efe954e198ed None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.068291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17873a62-18c1-4c5c-b529-b012aa8da75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.069228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17873a62-18c1-4c5c-b529-b012aa8da75f None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.069695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17873a62-18c1-4c5c-b529-b012aa8da75f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.070126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17873a62-18c1-4c5c-b529-b012aa8da75f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.071106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17873a62-18c1-4c5c-b529-b012aa8da75f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.072635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17873a62-18c1-4c5c-b529-b012aa8da75f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.152073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8a7d8-c89a-4344-94b0-b6c0ef96cd0a None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.152459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8a7d8-c89a-4344-94b0-b6c0ef96cd0a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.152685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8a7d8-c89a-4344-94b0-b6c0ef96cd0a None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.154209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 468/931] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:02.159842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba3312-ea7a-4694-890c-1da6330c8764 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.160135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba3312-ea7a-4694-890c-1da6330c8764 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.160444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba3312-ea7a-4694-890c-1da6330c8764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.179078 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-fbba3312-ea7a-4694-890c-1da6330c8764 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:02.180404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-fbba3312-ea7a-4694-890c-1da6330c8764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:02.183663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 464/932] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 23 15:38:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:38:02.195333 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7db2f2-ba6b-4dc8-a630-eef5b0610b0d None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.196092 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7db2f2-ba6b-4dc8-a630-eef5b0610b0d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.196667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7db2f2-ba6b-4dc8-a630-eef5b0610b0d None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.198960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 465/933] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:02.205549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c95d45-de53-460b-80d7-1f2999fed65e None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.207962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c95d45-de53-460b-80d7-1f2999fed65e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.207962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c95d45-de53-460b-80d7-1f2999fed65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.227322 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c7c95d45-de53-460b-80d7-1f2999fed65e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:02.229097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c7c95d45-de53-460b-80d7-1f2999fed65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:02.233476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:02.233618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:02.266869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 469/934] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:02.301405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba3312-ea7a-4694-890c-1da6330c8764 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.323804 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 466/935] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:02.345970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c95d45-de53-460b-80d7-1f2999fed65e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.421480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba3312-ea7a-4694-890c-1da6330c8764 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.472086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e8392-0ff1-4ff0-b019-23776d6c0de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.472933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e8392-0ff1-4ff0-b019-23776d6c0de4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.473227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e8392-0ff1-4ff0-b019-23776d6c0de4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.473502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e8392-0ff1-4ff0-b019-23776d6c0de4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.480169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c95d45-de53-460b-80d7-1f2999fed65e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.486498 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e8392-0ff1-4ff0-b019-23776d6c0de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-2c7e8392-0ff1-4ff0-b019-23776d6c0de4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:02.495124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:02.496121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:02.501709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 470/936] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 23 15:38:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:02.503044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e8392-0ff1-4ff0-b019-23776d6c0de4 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.534924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.535990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.536502 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.536954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.537859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.539629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.541422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: WARNING keystone.server.flask.common [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 15:38:02.546002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:02.550414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:02.605115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.sql.core [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 15:38:02.605115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 15:38:02.605115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [parameters: {'id': '80e1d08470fd496190b649213bc35ec4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 15:38:02.605115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87074) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 15:38:02.620409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e8392-0ff1-4ff0-b019-23776d6c0de4 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 80e1d08470fd496190b649213bc35ec4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 80e1d08470fd496190b649213bc35ec4. May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application conn.query(q) May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:38:02.629992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application result.read() May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 23 15:38:02.630829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return project_ref.to_dict() May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application next(self.gen) May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application yield resource May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application next(self.gen) May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self.session.rollback() May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self.force_reraise() May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise self.value May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application session.commit() May 23 15:38:02.631871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._prepare_impl() May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self.session.flush() May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._flush(objects) May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application compat.raise_( May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application flush_context.execute() May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 15:38:02.632818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rec.execute(self) May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 15:38:02.634452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application conn.query(q) May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application result.read() May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:38:02.637261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [parameters: {'id': '80e1d08470fd496190b649213bc35ec4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:02.640344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:02.643347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:02.648805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:02.648805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 23 15:38:02.648805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 23 15:38:02.648805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:02.648805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:02.648805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 23 15:38:02.648805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.Conflict( May 23 15:38:02.648805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 80e1d08470fd496190b649213bc35ec4. May 23 15:38:02.648805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:02.648805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 467/937] 10.222.0.23 () {60 vars in 1140 bytes} [Mon May 23 15:38:02 2022] POST /identity/v3/projects => generated 262 bytes in 302 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:02.654616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.666823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd26cba-c2c1-434b-bf04-9b381e93060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.667602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd26cba-c2c1-434b-bf04-9b381e93060a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.667968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd26cba-c2c1-434b-bf04-9b381e93060a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.668326 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd26cba-c2c1-434b-bf04-9b381e93060a None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.669674 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd26cba-c2c1-434b-bf04-9b381e93060a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:02.670068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd26cba-c2c1-434b-bf04-9b381e93060a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.672269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd26cba-c2c1-434b-bf04-9b381e93060a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.684524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 471/938] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 23 15:38:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:02.689749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd26cba-c2c1-434b-bf04-9b381e93060a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.788744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:02.789738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990daa3-b525-446c-acbd-c61abd19da20 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.798326 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd26cba-c2c1-434b-bf04-9b381e93060a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.836926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d509f-64cb-4983-b720-1b47d24d63e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.837306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145e554-fc7b-4482-bc63-26c4d150d75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:02.837647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d509f-64cb-4983-b720-1b47d24d63e7 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.837930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d509f-64cb-4983-b720-1b47d24d63e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.838204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d509f-64cb-4983-b720-1b47d24d63e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.839165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145e554-fc7b-4482-bc63-26c4d150d75a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:02.839658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145e554-fc7b-4482-bc63-26c4d150d75a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:02.840505 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145e554-fc7b-4482-bc63-26c4d150d75a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:02.849892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d509f-64cb-4983-b720-1b47d24d63e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.850448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145e554-fc7b-4482-bc63-26c4d150d75a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:02.851340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d509f-64cb-4983-b720-1b47d24d63e7 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.852710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145e554-fc7b-4482-bc63-26c4d150d75a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-835d509f-64cb-4983-b720-1b47d24d63e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:02.869428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:02.870366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:02.871266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:02.871266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:02.871266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:02.871266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-3145e554-fc7b-4482-bc63-26c4d150d75a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:38:02.872168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:02.873034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:02.873923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:38:02.873923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:38:02.873923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:38:02.873923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:38:02.873923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:38:02.873923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:38:02.873923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:02.873923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:02.875429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 472/939] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:02.876258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d509f-64cb-4983-b720-1b47d24d63e7 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.877128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 468/940] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:38:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:02.882935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145e554-fc7b-4482-bc63-26c4d150d75a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:02.942535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:02.984049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d509f-64cb-4983-b720-1b47d24d63e7 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:02.989681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145e554-fc7b-4482-bc63-26c4d150d75a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:03.025291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e2109-aa1c-4c32-9109-24cbaca7771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:03.025879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e2109-aa1c-4c32-9109-24cbaca7771f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:03.026352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e2109-aa1c-4c32-9109-24cbaca7771f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:03.026631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e2109-aa1c-4c32-9109-24cbaca7771f None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:03.027360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e2109-aa1c-4c32-9109-24cbaca7771f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:03.027674 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e2109-aa1c-4c32-9109-24cbaca7771f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:03.030615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e2109-aa1c-4c32-9109-24cbaca7771f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:03.033170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622487c3-217a-4e3a-aac6-9ba069cd5363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:03.034568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622487c3-217a-4e3a-aac6-9ba069cd5363 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:03.035317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622487c3-217a-4e3a-aac6-9ba069cd5363 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:03.036072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622487c3-217a-4e3a-aac6-9ba069cd5363 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:03.037766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622487c3-217a-4e3a-aac6-9ba069cd5363 None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:03.038617 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622487c3-217a-4e3a-aac6-9ba069cd5363 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:03.041580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622487c3-217a-4e3a-aac6-9ba069cd5363 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:03.047383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 473/941] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:03.048270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e2109-aa1c-4c32-9109-24cbaca7771f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:03.057517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:03.088605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 469/942] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:38:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:03.162437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e2109-aa1c-4c32-9109-24cbaca7771f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:03.199879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c16b3-9b7f-41c7-9a8e-5009214f2528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:03.200773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c16b3-9b7f-41c7-9a8e-5009214f2528 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:03.201055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c16b3-9b7f-41c7-9a8e-5009214f2528 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:03.201330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c16b3-9b7f-41c7-9a8e-5009214f2528 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:03.210622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c16b3-9b7f-41c7-9a8e-5009214f2528 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:03.212096 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c16b3-9b7f-41c7-9a8e-5009214f2528 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-086c16b3-9b7f-41c7-9a8e-5009214f2528 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:38:03.228255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:03.230501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:03.232560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 474/943] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 23 15:38:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:03.236516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622487c3-217a-4e3a-aac6-9ba069cd5363 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:03.358447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622487c3-217a-4e3a-aac6-9ba069cd5363 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:03.373358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:03.393992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2752a-80ef-4c12-a338-b862ff0fa341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:03.394665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2752a-80ef-4c12-a338-b862ff0fa341 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:03.394945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2752a-80ef-4c12-a338-b862ff0fa341 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:03.395220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2752a-80ef-4c12-a338-b862ff0fa341 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:03.396055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2752a-80ef-4c12-a338-b862ff0fa341 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:03.396425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2752a-80ef-4c12-a338-b862ff0fa341 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:03.397835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2752a-80ef-4c12-a338-b862ff0fa341 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:03.416469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 470/944] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 23 15:38:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:03.422942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c16b3-9b7f-41c7-9a8e-5009214f2528 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:03.495749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:03.531831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c16b3-9b7f-41c7-9a8e-5009214f2528 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:03.567849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e852d0-e66b-4d0d-b6e3-57764967b8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:03.568694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e852d0-e66b-4d0d-b6e3-57764967b8ab None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:03.568980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e852d0-e66b-4d0d-b6e3-57764967b8ab None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:03.569258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e852d0-e66b-4d0d-b6e3-57764967b8ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:03.580508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e852d0-e66b-4d0d-b6e3-57764967b8ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:03.581969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e852d0-e66b-4d0d-b6e3-57764967b8ab None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-72e852d0-e66b-4d0d-b6e3-57764967b8ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:03.599688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:03.600646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:03.601542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:03.601542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:03.601542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:03.601542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:03.605533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 475/945] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:03.606883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2752a-80ef-4c12-a338-b862ff0fa341 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:03.712751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2752a-80ef-4c12-a338-b862ff0fa341 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:03.755110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48b7d0-18cb-48c0-aadd-374fa96e3422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:03.755811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48b7d0-18cb-48c0-aadd-374fa96e3422 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:03.756094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48b7d0-18cb-48c0-aadd-374fa96e3422 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:03.756410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48b7d0-18cb-48c0-aadd-374fa96e3422 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:03.757142 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48b7d0-18cb-48c0-aadd-374fa96e3422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:03.757456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48b7d0-18cb-48c0-aadd-374fa96e3422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:03.758860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48b7d0-18cb-48c0-aadd-374fa96e3422 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:03.776567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 471/946] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:03.777904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e852d0-e66b-4d0d-b6e3-57764967b8ab None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:03.881734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e852d0-e66b-4d0d-b6e3-57764967b8ab None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:03.900032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:03.917712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca3a5f-0b95-4531-a56b-6a3cf767be4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:03.918581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca3a5f-0b95-4531-a56b-6a3cf767be4f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:03.928988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca3a5f-0b95-4531-a56b-6a3cf767be4f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:03.928988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca3a5f-0b95-4531-a56b-6a3cf767be4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:03.928988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca3a5f-0b95-4531-a56b-6a3cf767be4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:03.928988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca3a5f-0b95-4531-a56b-6a3cf767be4f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:03.931773 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4088-e9b1-4e80-bfe1-4746fc53092b None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:03.932232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4088-e9b1-4e80-bfe1-4746fc53092b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:03.932322 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4088-e9b1-4e80-bfe1-4746fc53092b None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:03.934367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 472/947] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:03.941920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc79544-33d4-43d1-b7b5-1be545153a5c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:03.942333 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc79544-33d4-43d1-b7b5-1be545153a5c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:03.942721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc79544-33d4-43d1-b7b5-1be545153a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-94ca3a5f-0b95-4531-a56b-6a3cf767be4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:38:03.946198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:03.947281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:03.948427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:38:03.948427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:38:03.948427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:38:03.948427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:38:03.948427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:38:03.948427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:38:03.948427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:03.948427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:03.950518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 476/948] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:38:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:03.957291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca3a5f-0b95-4531-a56b-6a3cf767be4f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:03.968187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9fc79544-33d4-43d1-b7b5-1be545153a5c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:03.971216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9fc79544-33d4-43d1-b7b5-1be545153a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:04.005926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:04.059581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 473/949] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:04.082734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc79544-33d4-43d1-b7b5-1be545153a5c None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:04.086965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca3a5f-0b95-4531-a56b-6a3cf767be4f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:04.133131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe10fce-0bfa-440a-a1d4-12e49e064694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:04.134657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe10fce-0bfa-440a-a1d4-12e49e064694 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:04.135163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe10fce-0bfa-440a-a1d4-12e49e064694 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:04.136091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe10fce-0bfa-440a-a1d4-12e49e064694 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:04.138368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe10fce-0bfa-440a-a1d4-12e49e064694 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:04.139361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe10fce-0bfa-440a-a1d4-12e49e064694 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:04.144079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe10fce-0bfa-440a-a1d4-12e49e064694 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:04.167098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 477/950] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:04.168905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe10fce-0bfa-440a-a1d4-12e49e064694 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:04.211140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc79544-33d4-43d1-b7b5-1be545153a5c None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:04.222904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:04.255476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27224e10-f6e5-4578-be96-87654b7e0d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:04.256299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27224e10-f6e5-4578-be96-87654b7e0d48 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:04.256626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27224e10-f6e5-4578-be96-87654b7e0d48 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:04.256901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27224e10-f6e5-4578-be96-87654b7e0d48 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:04.257554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27224e10-f6e5-4578-be96-87654b7e0d48 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:04.259387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27224e10-f6e5-4578-be96-87654b7e0d48 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-27224e10-f6e5-4578-be96-87654b7e0d48 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:04.275991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:04.277272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:04.284056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 474/951] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:38:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:04.295366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27224e10-f6e5-4578-be96-87654b7e0d48 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:04.307974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe10fce-0bfa-440a-a1d4-12e49e064694 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:04.330119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:04.355976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a25c87-e884-491d-b959-fcce841ab46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:04.357664 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a25c87-e884-491d-b959-fcce841ab46f None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:04.359073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a25c87-e884-491d-b959-fcce841ab46f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:04.359699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a25c87-e884-491d-b959-fcce841ab46f None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/72e29fe3a2b645248eef1d52217324d9/roles/8ad5ca9a5ad04e7fbb93d1330cf468e2` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:04.363182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a25c87-e884-491d-b959-fcce841ab46f None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=72e29fe3a2b645248eef1d52217324d9, role_id=8ad5ca9a5ad04e7fbb93d1330cf468e2)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:04.365615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a25c87-e884-491d-b959-fcce841ab46f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:04.417904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27224e10-f6e5-4578-be96-87654b7e0d48 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:04.447570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 478/952] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:38:04 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/72e29fe3a2b645248eef1d52217324d9/roles/8ad5ca9a5ad04e7fbb93d1330cf468e2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:38:04.454498 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec8d10-8741-4769-b27f-191c3449023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:04.455192 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec8d10-8741-4769-b27f-191c3449023d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:04.455477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec8d10-8741-4769-b27f-191c3449023d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:04.455757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec8d10-8741-4769-b27f-191c3449023d None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:04.456522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec8d10-8741-4769-b27f-191c3449023d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:04.456835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec8d10-8741-4769-b27f-191c3449023d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:04.458254 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec8d10-8741-4769-b27f-191c3449023d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:04.464447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 475/953] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 23 15:38:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:04.470634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a25c87-e884-491d-b959-fcce841ab46f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:04.579150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7a8ec-8544-4e37-a88a-de68a7d1586c None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:04.579491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7a8ec-8544-4e37-a88a-de68a7d1586c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:04.579720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7a8ec-8544-4e37-a88a-de68a7d1586c None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:04.581388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 476/954] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:04.585658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:04.586836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28a27-29db-44de-9103-65092b48a717 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:04.587128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28a27-29db-44de-9103-65092b48a717 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:04.587413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28a27-29db-44de-9103-65092b48a717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:04.589999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a25c87-e884-491d-b959-fcce841ab46f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:04.609612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e3c28a27-29db-44de-9103-65092b48a717 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:04.610896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e3c28a27-29db-44de-9103-65092b48a717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:04.628044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98218e6-6b12-422f-8251-9ca97dda5da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:04.628790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98218e6-6b12-422f-8251-9ca97dda5da6 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:04.629078 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98218e6-6b12-422f-8251-9ca97dda5da6 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:04.629359 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98218e6-6b12-422f-8251-9ca97dda5da6 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:04.630084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98218e6-6b12-422f-8251-9ca97dda5da6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:04.630397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98218e6-6b12-422f-8251-9ca97dda5da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:04.631868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98218e6-6b12-422f-8251-9ca97dda5da6 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:04.642170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 479/955] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 23 15:38:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:04.647281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98218e6-6b12-422f-8251-9ca97dda5da6 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:04.704577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 477/956] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:04.730843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c28a27-29db-44de-9103-65092b48a717 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:04.756774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:04.764511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98218e6-6b12-422f-8251-9ca97dda5da6 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:04.801201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ed2e2-4341-4aa8-a3f9-3a0c0e750d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:04.802652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ed2e2-4341-4aa8-a3f9-3a0c0e750d8d None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:04.803262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ed2e2-4341-4aa8-a3f9-3a0c0e750d8d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:04.803743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ed2e2-4341-4aa8-a3f9-3a0c0e750d8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:04.804727 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ed2e2-4341-4aa8-a3f9-3a0c0e750d8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:04.806320 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ed2e2-4341-4aa8-a3f9-3a0c0e750d8d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:04.839321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c28a27-29db-44de-9103-65092b48a717 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:04.874800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06055e2-f754-4bc7-9d00-a933e047bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:04.876150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06055e2-f754-4bc7-9d00-a933e047bb61 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:04.876457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06055e2-f754-4bc7-9d00-a933e047bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:04.876737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06055e2-f754-4bc7-9d00-a933e047bb61 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:04.879042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06055e2-f754-4bc7-9d00-a933e047bb61 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:04.885139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 480/957] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 23 15:38:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:38:04.885866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06055e2-f754-4bc7-9d00-a933e047bb61 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-e06055e2-f754-4bc7-9d00-a933e047bb61 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:04.903398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:04.904539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:04.905820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 478/958] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:38:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:04.911478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ed2e2-4341-4aa8-a3f9-3a0c0e750d8d None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:04.934160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:05.045779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ed2e2-4341-4aa8-a3f9-3a0c0e750d8d None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:05.089614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488b55d-4bf5-4926-9f6a-63a21f4b7a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:05.090213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488b55d-4bf5-4926-9f6a-63a21f4b7a8f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:05.090495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488b55d-4bf5-4926-9f6a-63a21f4b7a8f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:05.090774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488b55d-4bf5-4926-9f6a-63a21f4b7a8f None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:05.091487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488b55d-4bf5-4926-9f6a-63a21f4b7a8f None admin] RBAC: Adding query filter params (name=compute) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:05.091796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488b55d-4bf5-4926-9f6a-63a21f4b7a8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:05.093203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488b55d-4bf5-4926-9f6a-63a21f4b7a8f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:05.098694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 481/959] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:38:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:05.104335 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06055e2-f754-4bc7-9d00-a933e047bb61 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:05.195823 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:05.212952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06055e2-f754-4bc7-9d00-a933e047bb61 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:05.249699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bae262-d738-433b-b77c-594a01b53c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:05.250460 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bae262-d738-433b-b77c-594a01b53c19 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:05.250770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bae262-d738-433b-b77c-594a01b53c19 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:05.251098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bae262-d738-433b-b77c-594a01b53c19 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:05.251823 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bae262-d738-433b-b77c-594a01b53c19 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:05.252132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bae262-d738-433b-b77c-594a01b53c19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:05.253543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bae262-d738-433b-b77c-594a01b53c19 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:05.258667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 479/960] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:38:05 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:05.518042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5fbff-b6a5-4be5-bde6-7d3a22146476 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:05.518419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5fbff-b6a5-4be5-bde6-7d3a22146476 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:05.518632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5fbff-b6a5-4be5-bde6-7d3a22146476 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:05.520217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 482/961] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:05.525431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f85b83-30eb-4f88-98b8-4585b63a8a1a None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:05.525724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f85b83-30eb-4f88-98b8-4585b63a8a1a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:05.525999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f85b83-30eb-4f88-98b8-4585b63a8a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:05.543650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c7f85b83-30eb-4f88-98b8-4585b63a8a1a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:05.544847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c7f85b83-30eb-4f88-98b8-4585b63a8a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:05.617728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 480/962] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:05.642378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5fbff-b6a5-4be5-bde6-7d3a22146476 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:05.726247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:05.748923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5fbff-b6a5-4be5-bde6-7d3a22146476 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:05.786655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:05.787253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:05.787532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:05.787801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:05.788528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:05.789957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:05.791269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WARNING keystone.server.flask.common [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 15:38:05.855888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.sql.core [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 15:38:05.855888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 15:38:05.855888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [parameters: {'user_id': '7f15a379700b4187b60a140fced62ef3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 15:38:05.855888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87073) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application conn.query(q) May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application result.read() May 23 15:38:05.863726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:38:05.864628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application next(self.gen) May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application yield resource May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application next(self.gen) May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self.session.rollback() May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self.force_reraise() May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise self.value May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application session.commit() May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 15:38:05.865595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._prepare_impl() May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self.session.flush() May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._flush(objects) May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application compat.raise_( May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application flush_context.execute() May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rec.execute(self) May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 15:38:05.867276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application result = connection._execute_20( May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 15:38:05.869474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application result = self._query(query) May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application conn.query(q) May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application result.read() May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 15:38:05.871617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [parameters: {'user_id': '7f15a379700b4187b60a140fced62ef3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:05.876732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 23 15:38:05.878699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 23 15:38:05.880919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 23 15:38:05.880919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 23 15:38:05.880919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 15:38:05.880919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 15:38:05.880919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 15:38:05.880919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:05.880919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 483/963] 10.222.0.23 () {60 vars in 1131 bytes} [Mon May 23 15:38:05 2022] POST /identity/v3/users => generated 160 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:05.881824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f85b83-30eb-4f88-98b8-4585b63a8a1a None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:05.962502 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:05.984893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f85b83-30eb-4f88-98b8-4585b63a8a1a None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:06.020993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52c640-16af-4e01-9608-ba13c9126218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:06.022119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52c640-16af-4e01-9608-ba13c9126218 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:06.022717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52c640-16af-4e01-9608-ba13c9126218 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:06.023200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52c640-16af-4e01-9608-ba13c9126218 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:06.030925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52c640-16af-4e01-9608-ba13c9126218 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:06.032672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52c640-16af-4e01-9608-ba13c9126218 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-fc52c640-16af-4e01-9608-ba13c9126218 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:38:06.048053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:06.049055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:06.050044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:06.050044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:06.050044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:38:06.050044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:38:06.050044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:38:06.050044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:38:06.050044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:38:06.050044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:38:06.050044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 15:38:06.050044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:06.052711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 481/964] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:38:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:06.058903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:06.168545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be06fe9-a22d-4edf-8ed6-8e2c40c815ad None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:06.204529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d774fb-5c53-4029-886d-cccf57e21f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:06.205484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d774fb-5c53-4029-886d-cccf57e21f85 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:06.205912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d774fb-5c53-4029-886d-cccf57e21f85 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:06.206482 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d774fb-5c53-4029-886d-cccf57e21f85 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:06.207521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d774fb-5c53-4029-886d-cccf57e21f85 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:06.207970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d774fb-5c53-4029-886d-cccf57e21f85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:06.209906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d774fb-5c53-4029-886d-cccf57e21f85 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:06.234366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:06.249252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 484/965] 10.222.0.23 () {56 vars in 1101 bytes} [Mon May 23 15:38:06 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:06.804325 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24538cd-8d86-4be9-8904-22e04614a9d2 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:06.804906 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24538cd-8d86-4be9-8904-22e04614a9d2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:06.804906 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24538cd-8d86-4be9-8904-22e04614a9d2 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:06.806290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 482/966] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:06.811766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73d042-2cfb-47b5-aebc-a956ff5d02fb None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:06.812061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73d042-2cfb-47b5-aebc-a956ff5d02fb None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:06.812342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73d042-2cfb-47b5-aebc-a956ff5d02fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:06.829455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0a73d042-2cfb-47b5-aebc-a956ff5d02fb None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:06.830950 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0a73d042-2cfb-47b5-aebc-a956ff5d02fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:06.900139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 485/967] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:06.923744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24538cd-8d86-4be9-8904-22e04614a9d2 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.025951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24538cd-8d86-4be9-8904-22e04614a9d2 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:07.064258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09177f-6f50-4119-a613-c7cd80e7e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:07.065018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09177f-6f50-4119-a613-c7cd80e7e1e2 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.065296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09177f-6f50-4119-a613-c7cd80e7e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.065571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09177f-6f50-4119-a613-c7cd80e7e1e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.072490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:07.076882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09177f-6f50-4119-a613-c7cd80e7e1e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-1e09177f-6f50-4119-a613-c7cd80e7e1e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:07.085627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:07.093006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:07.093882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 483/968] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 23 15:38:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:07.095278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73d042-2cfb-47b5-aebc-a956ff5d02fb None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.197398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73d042-2cfb-47b5-aebc-a956ff5d02fb None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:07.225358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:07.225754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541701a2-5334-4139-b8dd-e2d8bf5aa5fc None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.226044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541701a2-5334-4139-b8dd-e2d8bf5aa5fc None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.226324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541701a2-5334-4139-b8dd-e2d8bf5aa5fc None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.227706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 484/969] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:07.232880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bedd5-dbee-4e16-a24b-6af8826c4c18 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.233171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bedd5-dbee-4e16-a24b-6af8826c4c18 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.233446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bedd5-dbee-4e16-a24b-6af8826c4c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.234264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab589b2e-e7fd-41d7-a97f-146205dd2df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:07.234965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab589b2e-e7fd-41d7-a97f-146205dd2df3 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.235965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab589b2e-e7fd-41d7-a97f-146205dd2df3 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.235965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab589b2e-e7fd-41d7-a97f-146205dd2df3 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.236309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab589b2e-e7fd-41d7-a97f-146205dd2df3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:07.236641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab589b2e-e7fd-41d7-a97f-146205dd2df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:07.238015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab589b2e-e7fd-41d7-a97f-146205dd2df3 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:07.247970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 486/970] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 23 15:38:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:07.250766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b85bedd5-dbee-4e16-a24b-6af8826c4c18 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:07.251958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b85bedd5-dbee-4e16-a24b-6af8826c4c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:07.253239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab589b2e-e7fd-41d7-a97f-146205dd2df3 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.327827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 485/971] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:07.351726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bedd5-dbee-4e16-a24b-6af8826c4c18 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.362267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab589b2e-e7fd-41d7-a97f-146205dd2df3 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:07.399796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28a529-a735-4181-9dfb-84e8b2657099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:07.400627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28a529-a735-4181-9dfb-84e8b2657099 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.400910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28a529-a735-4181-9dfb-84e8b2657099 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.401184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28a529-a735-4181-9dfb-84e8b2657099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.414127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28a529-a735-4181-9dfb-84e8b2657099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:07.415586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28a529-a735-4181-9dfb-84e8b2657099 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-8d28a529-a735-4181-9dfb-84e8b2657099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:07.433764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:07.434769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:07.435690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:07.435690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:07.435690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:07.435690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:07.437598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 487/972] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:07.443226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28a529-a735-4181-9dfb-84e8b2657099 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.475347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bedd5-dbee-4e16-a24b-6af8826c4c18 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:07.491168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:07.506088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:07.514655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1e9f4-5a04-40ef-8d50-3a3cd671281a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:07.515467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1e9f4-5a04-40ef-8d50-3a3cd671281a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.515749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1e9f4-5a04-40ef-8d50-3a3cd671281a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.516070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1e9f4-5a04-40ef-8d50-3a3cd671281a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.517269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1e9f4-5a04-40ef-8d50-3a3cd671281a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:07.518638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1e9f4-5a04-40ef-8d50-3a3cd671281a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-85e1e9f4-5a04-40ef-8d50-3a3cd671281a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:07.532927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:07.534342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:07.535176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 486/973] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:38:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:07.540132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1e9f4-5a04-40ef-8d50-3a3cd671281a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.560965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28a529-a735-4181-9dfb-84e8b2657099 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:07.595701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e60d50-ab3d-4961-af61-857b3f9554cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:07.596291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e60d50-ab3d-4961-af61-857b3f9554cd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.596609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e60d50-ab3d-4961-af61-857b3f9554cd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.596890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e60d50-ab3d-4961-af61-857b3f9554cd None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.597618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e60d50-ab3d-4961-af61-857b3f9554cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:07.597932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e60d50-ab3d-4961-af61-857b3f9554cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:07.599303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e60d50-ab3d-4961-af61-857b3f9554cd None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:07.610865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 488/974] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:07.612077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd939e-4013-44fd-bc3f-30d94ac54e7f None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.612383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd939e-4013-44fd-bc3f-30d94ac54e7f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.612679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd939e-4013-44fd-bc3f-30d94ac54e7f None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.613901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 489/975] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:07.616189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd939e-4013-44fd-bc3f-30d94ac54e7f None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.643493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1e9f4-5a04-40ef-8d50-3a3cd671281a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:07.676299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bad78-944f-4e2f-adf9-f3b43b3e8bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:07.676985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bad78-944f-4e2f-adf9-f3b43b3e8bae None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.677273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bad78-944f-4e2f-adf9-f3b43b3e8bae None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.677548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bad78-944f-4e2f-adf9-f3b43b3e8bae None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.678250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bad78-944f-4e2f-adf9-f3b43b3e8bae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:07.678555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bad78-944f-4e2f-adf9-f3b43b3e8bae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:07.679908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bad78-944f-4e2f-adf9-f3b43b3e8bae None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:07.684696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 487/976] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 23 15:38:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:07.685843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed038331-f835-4526-86de-634aa02e8cb8 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.686135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed038331-f835-4526-86de-634aa02e8cb8 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.686420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed038331-f835-4526-86de-634aa02e8cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.702541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ed038331-f835-4526-86de-634aa02e8cb8 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:07.704302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ed038331-f835-4526-86de-634aa02e8cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:07.716548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd939e-4013-44fd-bc3f-30d94ac54e7f None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:07.718055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:07.751666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55637e8-9cc9-492a-8b34-fcb68abf9e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:07.752505 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55637e8-9cc9-492a-8b34-fcb68abf9e05 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.752794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55637e8-9cc9-492a-8b34-fcb68abf9e05 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.753071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55637e8-9cc9-492a-8b34-fcb68abf9e05 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.753757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:07.760466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55637e8-9cc9-492a-8b34-fcb68abf9e05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:07.761981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55637e8-9cc9-492a-8b34-fcb68abf9e05 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:07.770489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 488/977] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:07.773480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed038331-f835-4526-86de-634aa02e8cb8 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-c55637e8-9cc9-492a-8b34-fcb68abf9e05 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:38:07.779303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:07.780465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:07.781453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 490/978] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 23 15:38:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:07.786178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55637e8-9cc9-492a-8b34-fcb68abf9e05 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.874735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed038331-f835-4526-86de-634aa02e8cb8 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:07.885015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55637e8-9cc9-492a-8b34-fcb68abf9e05 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:07.908937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd977ec-54b3-414e-9c05-a7bd54595bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:07.909604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd977ec-54b3-414e-9c05-a7bd54595bbe None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.909890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd977ec-54b3-414e-9c05-a7bd54595bbe None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.910171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd977ec-54b3-414e-9c05-a7bd54595bbe None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.910875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd977ec-54b3-414e-9c05-a7bd54595bbe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:07.911186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd977ec-54b3-414e-9c05-a7bd54595bbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:07.912575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd977ec-54b3-414e-9c05-a7bd54595bbe None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:07.917490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 489/979] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 23 15:38:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:07.918074 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd977ec-54b3-414e-9c05-a7bd54595bbe None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.923271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b80dc-aca5-4a7b-8f9a-f1f6375f81fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:07.923948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b80dc-aca5-4a7b-8f9a-f1f6375f81fe None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:07.924233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b80dc-aca5-4a7b-8f9a-f1f6375f81fe None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:07.924544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b80dc-aca5-4a7b-8f9a-f1f6375f81fe None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:07.925385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b80dc-aca5-4a7b-8f9a-f1f6375f81fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:07.925695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b80dc-aca5-4a7b-8f9a-f1f6375f81fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:07.927116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b80dc-aca5-4a7b-8f9a-f1f6375f81fe None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:07.944529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 491/980] 10.222.0.23 () {56 vars in 1149 bytes} [Mon May 23 15:38:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:07.945314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b80dc-aca5-4a7b-8f9a-f1f6375f81fe None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:07.966036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:07.976064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:08.025436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd977ec-54b3-414e-9c05-a7bd54595bbe None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:08.043286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b80dc-aca5-4a7b-8f9a-f1f6375f81fe None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:08.058582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e33f0-a4e7-479c-9af9-80182245fa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:08.059281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e33f0-a4e7-479c-9af9-80182245fa56 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:08.059569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e33f0-a4e7-479c-9af9-80182245fa56 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:08.059849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e33f0-a4e7-479c-9af9-80182245fa56 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:08.060604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e33f0-a4e7-479c-9af9-80182245fa56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:08.061958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e33f0-a4e7-479c-9af9-80182245fa56 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:08.077369 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6856953-1498-4d9d-936e-15371cc356de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:08.080593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6856953-1498-4d9d-936e-15371cc356de None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:08.080593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6856953-1498-4d9d-936e-15371cc356de None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:08.080593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6856953-1498-4d9d-936e-15371cc356de None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:08.080593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6856953-1498-4d9d-936e-15371cc356de None admin] RBAC: Adding query filter params (interface=admin, service_id=1548b5d72c6c4d23a5938af8a417d591, region_id=RegionOne) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:08.080593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6856953-1498-4d9d-936e-15371cc356de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:08.081217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6856953-1498-4d9d-936e-15371cc356de None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:08.086284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 492/981] 10.222.0.23 () {56 vars in 1249 bytes} [Mon May 23 15:38:07 2022] GET /identity/v3/endpoints?service_id=1548b5d72c6c4d23a5938af8a417d591&interface=admin®ion_id=RegionOne => generated 193 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:08.087107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6856953-1498-4d9d-936e-15371cc356de None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:08.125139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 490/982] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:38:07 2022] POST /identity/v3/services => generated 242 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:38:08.126892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e33f0-a4e7-479c-9af9-80182245fa56 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:08.184822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:08.203161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6856953-1498-4d9d-936e-15371cc356de None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:08.250427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f78ca-73cc-4f86-9be9-a9dbba54fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:08.250827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e33f0-a4e7-479c-9af9-80182245fa56 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:08.251113 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f78ca-73cc-4f86-9be9-a9dbba54fa82 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:08.251400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f78ca-73cc-4f86-9be9-a9dbba54fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:08.251677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f78ca-73cc-4f86-9be9-a9dbba54fa82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:08.258573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:08.292228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b3ed8-29bb-41d9-b4f8-b44e50bd865e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:08.293166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b3ed8-29bb-41d9-b4f8-b44e50bd865e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:08.293618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b3ed8-29bb-41d9-b4f8-b44e50bd865e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:08.293898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b3ed8-29bb-41d9-b4f8-b44e50bd865e None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:08.294659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b3ed8-29bb-41d9-b4f8-b44e50bd865e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:08.295996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b3ed8-29bb-41d9-b4f8-b44e50bd865e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:08.304024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 491/983] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 23 15:38:08 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:38:08.337841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f78ca-73cc-4f86-9be9-a9dbba54fa82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:08.339626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f78ca-73cc-4f86-9be9-a9dbba54fa82 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:08.431600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:08.452758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 493/984] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:08 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:38:08.458842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b3ed8-29bb-41d9-b4f8-b44e50bd865e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:08.564936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b3ed8-29bb-41d9-b4f8-b44e50bd865e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:08.600088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee70dd5-9c77-4829-a7c8-bbb6fd66b014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:08.600914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee70dd5-9c77-4829-a7c8-bbb6fd66b014 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:08.601202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee70dd5-9c77-4829-a7c8-bbb6fd66b014 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:08.601480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee70dd5-9c77-4829-a7c8-bbb6fd66b014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:08.612529 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee70dd5-9c77-4829-a7c8-bbb6fd66b014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:08.613971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee70dd5-9c77-4829-a7c8-bbb6fd66b014 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-0ee70dd5-9c77-4829-a7c8-bbb6fd66b014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:08.631069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:08.631969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:08.633366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:08.633366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:08.633366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:08.633366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:08.633366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 492/985] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:08.638393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f78ca-73cc-4f86-9be9-a9dbba54fa82 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:08.739796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f78ca-73cc-4f86-9be9-a9dbba54fa82 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:08.773363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbab856-5fd6-48ff-aa1d-a2c4262cbb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:08.774024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab856-5fd6-48ff-aa1d-a2c4262cbb3f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:08.774305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab856-5fd6-48ff-aa1d-a2c4262cbb3f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:08.774578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab856-5fd6-48ff-aa1d-a2c4262cbb3f None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:08.775292 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbab856-5fd6-48ff-aa1d-a2c4262cbb3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:08.775605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbab856-5fd6-48ff-aa1d-a2c4262cbb3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:08.776991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbab856-5fd6-48ff-aa1d-a2c4262cbb3f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:08.792452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 494/986] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:08.794536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee70dd5-9c77-4829-a7c8-bbb6fd66b014 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:08.819888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:08.894938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee70dd5-9c77-4829-a7c8-bbb6fd66b014 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:08.928712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d249ad-3998-40e9-847b-9ec3097d990f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:08.929512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d249ad-3998-40e9-847b-9ec3097d990f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:08.929793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d249ad-3998-40e9-847b-9ec3097d990f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:08.930067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d249ad-3998-40e9-847b-9ec3097d990f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:08.936797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d249ad-3998-40e9-847b-9ec3097d990f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:08.938830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d249ad-3998-40e9-847b-9ec3097d990f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-99d249ad-3998-40e9-847b-9ec3097d990f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:38:08.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:08.953263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:08.954849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:38:08.954849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:38:08.954849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:38:08.954849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:38:08.954849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:38:08.954849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:38:08.954849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:08.954849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:08.955900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 493/987] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:08.959651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbab856-5fd6-48ff-aa1d-a2c4262cbb3f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:09.002365 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:09.066360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbab856-5fd6-48ff-aa1d-a2c4262cbb3f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:09.099610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b8de1-bdf9-4f92-a10e-25bfb403f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:09.100277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8de1-bdf9-4f92-a10e-25bfb403f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:09.100589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8de1-bdf9-4f92-a10e-25bfb403f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:09.100869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8de1-bdf9-4f92-a10e-25bfb403f2d6 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:09.101655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b8de1-bdf9-4f92-a10e-25bfb403f2d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:09.101969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b8de1-bdf9-4f92-a10e-25bfb403f2d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:09.103401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b8de1-bdf9-4f92-a10e-25bfb403f2d6 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:09.120140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 495/988] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:09.121359 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d249ad-3998-40e9-847b-9ec3097d990f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:09.274430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:09.300503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d249ad-3998-40e9-847b-9ec3097d990f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:09.335651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08318eae-58b4-43a0-b8bd-9d91b9a74b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:09.336305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08318eae-58b4-43a0-b8bd-9d91b9a74b2c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:09.336684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08318eae-58b4-43a0-b8bd-9d91b9a74b2c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:09.336963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08318eae-58b4-43a0-b8bd-9d91b9a74b2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:09.337793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08318eae-58b4-43a0-b8bd-9d91b9a74b2c None admin] RBAC: Adding query filter params (role.id=8ad5ca9a5ad04e7fbb93d1330cf468e2, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=72e29fe3a2b645248eef1d52217324d9) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:09.338173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08318eae-58b4-43a0-b8bd-9d91b9a74b2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:09.339597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08318eae-58b4-43a0-b8bd-9d91b9a74b2c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:09.352393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 494/989] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:38:09 2022] GET /identity/v3/role_assignments?user.id=72e29fe3a2b645248eef1d52217324d9&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=8ad5ca9a5ad04e7fbb93d1330cf468e2 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:10.377858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce68737-c046-471b-a413-91c544ba9633 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:10.378395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce68737-c046-471b-a413-91c544ba9633 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:10.378395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce68737-c046-471b-a413-91c544ba9633 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:10.379908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 496/990] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:10.384905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dfad0-f403-4154-bd35-9ee5358bb387 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:10.385279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dfad0-f403-4154-bd35-9ee5358bb387 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:10.385467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dfad0-f403-4154-bd35-9ee5358bb387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:10.402462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-657dfad0-f403-4154-bd35-9ee5358bb387 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:10.403630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-657dfad0-f403-4154-bd35-9ee5358bb387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:10.472547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 495/991] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:10.493714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce68737-c046-471b-a413-91c544ba9633 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:10.539689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176f0bd-d2b0-4807-9475-012491bf9360 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:10.539881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176f0bd-d2b0-4807-9475-012491bf9360 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:10.540165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176f0bd-d2b0-4807-9475-012491bf9360 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:10.541581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 496/992] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:10.546752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9795b-949d-435f-9b0e-a23ca8acfe7b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:10.547041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9795b-949d-435f-9b0e-a23ca8acfe7b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:10.547348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9795b-949d-435f-9b0e-a23ca8acfe7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:10.564777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b2c9795b-949d-435f-9b0e-a23ca8acfe7b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:10.565944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b2c9795b-949d-435f-9b0e-a23ca8acfe7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:10.598923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:10.603040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce68737-c046-471b-a413-91c544ba9633 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:10.638079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018f456-8dc8-4429-87c3-e394eba04e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:10.638877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f456-8dc8-4429-87c3-e394eba04e45 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:10.639161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f456-8dc8-4429-87c3-e394eba04e45 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:10.639446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f456-8dc8-4429-87c3-e394eba04e45 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:10.640089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018f456-8dc8-4429-87c3-e394eba04e45 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:10.641645 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018f456-8dc8-4429-87c3-e394eba04e45 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:10.647505 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 497/993] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-0018f456-8dc8-4429-87c3-e394eba04e45 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:10.653337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:10.654313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:10.655141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 497/994] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:38:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:10.663412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9795b-949d-435f-9b0e-a23ca8acfe7b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:10.669794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:10.672018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018f456-8dc8-4429-87c3-e394eba04e45 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:10.767341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9795b-949d-435f-9b0e-a23ca8acfe7b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:10.773583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018f456-8dc8-4429-87c3-e394eba04e45 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:10.802587 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6c414-4a9f-4357-866a-8fa83db0e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:10.803300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6c414-4a9f-4357-866a-8fa83db0e020 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:10.803581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6c414-4a9f-4357-866a-8fa83db0e020 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:10.803854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6c414-4a9f-4357-866a-8fa83db0e020 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:10.804581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6c414-4a9f-4357-866a-8fa83db0e020 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:10.804940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6c414-4a9f-4357-866a-8fa83db0e020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:10.806228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6c414-4a9f-4357-866a-8fa83db0e020 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:10.807897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f723a-8c8b-4125-9a6e-8e1cacb464e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:10.808720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f723a-8c8b-4125-9a6e-8e1cacb464e4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:10.809000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f723a-8c8b-4125-9a6e-8e1cacb464e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:10.809267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f723a-8c8b-4125-9a6e-8e1cacb464e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:10.809903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f723a-8c8b-4125-9a6e-8e1cacb464e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:10.811249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f723a-8c8b-4125-9a6e-8e1cacb464e4 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:10.816750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 498/995] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:38:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:10.817782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6c414-4a9f-4357-866a-8fa83db0e020 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-830f723a-8c8b-4125-9a6e-8e1cacb464e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:10.823209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:10.824193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 15:38:10.825097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:10.830134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 498/996] 10.222.0.23 () {56 vars in 1126 bytes} [Mon May 23 15:38:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:10.830806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f723a-8c8b-4125-9a6e-8e1cacb464e4 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:10.862825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:10.893751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:10.925976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6c414-4a9f-4357-866a-8fa83db0e020 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:10.941350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f723a-8c8b-4125-9a6e-8e1cacb464e4 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:10.961898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c258f0-f947-4dd4-bf9c-256c32d4d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:10.962568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c258f0-f947-4dd4-bf9c-256c32d4d773 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:10.962858 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c258f0-f947-4dd4-bf9c-256c32d4d773 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:10.963136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c258f0-f947-4dd4-bf9c-256c32d4d773 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:10.963849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c258f0-f947-4dd4-bf9c-256c32d4d773 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:10.964159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c258f0-f947-4dd4-bf9c-256c32d4d773 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:10.965682 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c258f0-f947-4dd4-bf9c-256c32d4d773 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:10.975765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 499/997] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:38:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:10.977009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c258f0-f947-4dd4-bf9c-256c32d4d773 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:10.979244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417ae8d-d27a-4b72-a95f-fb35e9567a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:10.979919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417ae8d-d27a-4b72-a95f-fb35e9567a77 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:10.980201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417ae8d-d27a-4b72-a95f-fb35e9567a77 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:10.980512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417ae8d-d27a-4b72-a95f-fb35e9567a77 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:10.981222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417ae8d-d27a-4b72-a95f-fb35e9567a77 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:10.981528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417ae8d-d27a-4b72-a95f-fb35e9567a77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:10.982917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417ae8d-d27a-4b72-a95f-fb35e9567a77 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:10.987759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 499/998] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 23 15:38:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:10.993141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417ae8d-d27a-4b72-a95f-fb35e9567a77 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:11.087918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c258f0-f947-4dd4-bf9c-256c32d4d773 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:11.096233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:11.104724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417ae8d-d27a-4b72-a95f-fb35e9567a77 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:11.123543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8126c24-4f4b-4174-b752-35316240d796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:11.124576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8126c24-4f4b-4174-b752-35316240d796 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:11.125095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8126c24-4f4b-4174-b752-35316240d796 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:11.125456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8126c24-4f4b-4174-b752-35316240d796 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:11.126341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8126c24-4f4b-4174-b752-35316240d796 None admin] RBAC: Adding query filter params (interface=public, service_id=ec8274ef83ab40318cbb6f09c6a5bcf5, region_id=RegionOne) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:11.126612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8126c24-4f4b-4174-b752-35316240d796 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:11.127918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8126c24-4f4b-4174-b752-35316240d796 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:11.130055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:11.135707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 500/999] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 23 15:38:10 2022] GET /identity/v3/endpoints?service_id=ec8274ef83ab40318cbb6f09c6a5bcf5&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:11.140969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528168fa-244a-498f-a1e3-84c1e9c4f4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:11.141650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8126c24-4f4b-4174-b752-35316240d796 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:11.142162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528168fa-244a-498f-a1e3-84c1e9c4f4df None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:11.142444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528168fa-244a-498f-a1e3-84c1e9c4f4df None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:11.142712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528168fa-244a-498f-a1e3-84c1e9c4f4df None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:11.143415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528168fa-244a-498f-a1e3-84c1e9c4f4df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:11.143717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528168fa-244a-498f-a1e3-84c1e9c4f4df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:11.145736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528168fa-244a-498f-a1e3-84c1e9c4f4df None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:11.150213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 500/1000] 10.222.0.23 () {56 vars in 1122 bytes} [Mon May 23 15:38:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:11.155616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528168fa-244a-498f-a1e3-84c1e9c4f4df None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:11.244038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8126c24-4f4b-4174-b752-35316240d796 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:11.261119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528168fa-244a-498f-a1e3-84c1e9c4f4df None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:11.282339 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1adcd-d6cf-4dbc-9e18-ec933103136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:11.284223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1adcd-d6cf-4dbc-9e18-ec933103136c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:11.284770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1adcd-d6cf-4dbc-9e18-ec933103136c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:11.285528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1adcd-d6cf-4dbc-9e18-ec933103136c None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:11.286622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1adcd-d6cf-4dbc-9e18-ec933103136c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:11.289323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1adcd-d6cf-4dbc-9e18-ec933103136c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:11.296178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 501/1001] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 23 15:38:11 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:38:11.306713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22492fa1-6bb0-4c5a-a7c9-dd18eaa65b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:11.307405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492fa1-6bb0-4c5a-a7c9-dd18eaa65b5e None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:11.307688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492fa1-6bb0-4c5a-a7c9-dd18eaa65b5e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:11.307960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22492fa1-6bb0-4c5a-a7c9-dd18eaa65b5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:11.308917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22492fa1-6bb0-4c5a-a7c9-dd18eaa65b5e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:11.310345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22492fa1-6bb0-4c5a-a7c9-dd18eaa65b5e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:11.384558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 501/1002] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 23 15:38:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:38:11.595704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7d0f7-e566-4c65-9c29-127fcdfad038 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:11.596221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7d0f7-e566-4c65-9c29-127fcdfad038 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:11.596308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7d0f7-e566-4c65-9c29-127fcdfad038 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:11.597804 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 502/1003] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:11.602895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0648e-0137-4c23-9bcb-34b3363b9dac None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:11.603189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0648e-0137-4c23-9bcb-34b3363b9dac None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:11.603469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0648e-0137-4c23-9bcb-34b3363b9dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:11.620866 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b2e0648e-0137-4c23-9bcb-34b3363b9dac None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:11.622072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b2e0648e-0137-4c23-9bcb-34b3363b9dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:11.634360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:11.699022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 502/1004] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:11 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:11.720830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7d0f7-e566-4c65-9c29-127fcdfad038 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:11.786290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:11.830209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7d0f7-e566-4c65-9c29-127fcdfad038 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:11.863969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64313c7e-feb3-49b4-96be-f1d2821fa83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:11.864808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64313c7e-feb3-49b4-96be-f1d2821fa83f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:11.865096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64313c7e-feb3-49b4-96be-f1d2821fa83f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:11.865372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64313c7e-feb3-49b4-96be-f1d2821fa83f None admin] PATH_INFO: `/v3/services/image` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:11.866025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64313c7e-feb3-49b4-96be-f1d2821fa83f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:11.867383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64313c7e-feb3-49b4-96be-f1d2821fa83f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-64313c7e-feb3-49b4-96be-f1d2821fa83f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:11.879471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:11.880748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:38:11.881600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:11.884438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 503/1005] 10.222.0.23 () {56 vars in 1105 bytes} [Mon May 23 15:38:11 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:11.886752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0648e-0137-4c23-9bcb-34b3363b9dac None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:11.990322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0648e-0137-4c23-9bcb-34b3363b9dac None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:12.026593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a9f74-8d2d-40a3-b6e9-2e9cd920bf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:12.027309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a9f74-8d2d-40a3-b6e9-2e9cd920bf6e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:12.027604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a9f74-8d2d-40a3-b6e9-2e9cd920bf6e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:12.027883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a9f74-8d2d-40a3-b6e9-2e9cd920bf6e None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:12.028624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a9f74-8d2d-40a3-b6e9-2e9cd920bf6e None admin] RBAC: Adding query filter params (name=image) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:12.028939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a9f74-8d2d-40a3-b6e9-2e9cd920bf6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:12.030315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a9f74-8d2d-40a3-b6e9-2e9cd920bf6e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:12.040156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 503/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:12.041431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64313c7e-feb3-49b4-96be-f1d2821fa83f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:12.144738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64313c7e-feb3-49b4-96be-f1d2821fa83f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:12.171374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:12.180987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383221a-8b39-4f8e-9472-a0ce1c136486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:12.181657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383221a-8b39-4f8e-9472-a0ce1c136486 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:12.181943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383221a-8b39-4f8e-9472-a0ce1c136486 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:12.182223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383221a-8b39-4f8e-9472-a0ce1c136486 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:12.182936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383221a-8b39-4f8e-9472-a0ce1c136486 None admin] RBAC: Adding query filter params (type=image) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:12.183307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383221a-8b39-4f8e-9472-a0ce1c136486 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:12.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383221a-8b39-4f8e-9472-a0ce1c136486 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:12.194186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 504/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:13.507807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676b34f-63b3-471a-a1a4-aa258a960f97 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:13.508311 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676b34f-63b3-471a-a1a4-aa258a960f97 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:13.508609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676b34f-63b3-471a-a1a4-aa258a960f97 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:13.509873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 504/1008] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:13.515184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fba4fe-b8c9-459b-b094-b411b82b5fe7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:13.515556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fba4fe-b8c9-459b-b094-b411b82b5fe7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:13.515737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fba4fe-b8c9-459b-b094-b411b82b5fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:13.533343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-98fba4fe-b8c9-459b-b094-b411b82b5fe7 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:13.534516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-98fba4fe-b8c9-459b-b094-b411b82b5fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:13.602027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 505/1009] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:13 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:13.624347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676b34f-63b3-471a-a1a4-aa258a960f97 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:13.626402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fb42c-ca33-4d4b-956e-ea5779704524 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:13.626699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fb42c-ca33-4d4b-956e-ea5779704524 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:13.626987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fb42c-ca33-4d4b-956e-ea5779704524 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:13.628351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 506/1010] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:13.633289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ea2cd-5e23-49ac-bf06-e78bd13c3969 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:13.633645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ea2cd-5e23-49ac-bf06-e78bd13c3969 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:13.633810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:13.633911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ea2cd-5e23-49ac-bf06-e78bd13c3969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:13.651578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-994ea2cd-5e23-49ac-bf06-e78bd13c3969 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:13.652790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-994ea2cd-5e23-49ac-bf06-e78bd13c3969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:13.720408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 507/1011] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:13 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:13.734532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676b34f-63b3-471a-a1a4-aa258a960f97 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:13.743883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ea2cd-5e23-49ac-bf06-e78bd13c3969 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:13.775958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcfae5-6ac6-48ef-a3cb-e384b5dc02b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:13.776788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcfae5-6ac6-48ef-a3cb-e384b5dc02b9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:13.777075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcfae5-6ac6-48ef-a3cb-e384b5dc02b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:13.777513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fcfae5-6ac6-48ef-a3cb-e384b5dc02b9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:13.778910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcfae5-6ac6-48ef-a3cb-e384b5dc02b9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:13.780284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fcfae5-6ac6-48ef-a3cb-e384b5dc02b9 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:13.793828 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-58fcfae5-6ac6-48ef-a3cb-e384b5dc02b9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:13.795340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:13.796316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:13.802474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 505/1012] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:38:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:13.804911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcfae5-6ac6-48ef-a3cb-e384b5dc02b9 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:13.847810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ea2cd-5e23-49ac-bf06-e78bd13c3969 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:13.880673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:13.883099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d8641-5394-4b3a-b558-8a0fabdf4f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:13.883906 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d8641-5394-4b3a-b558-8a0fabdf4f7c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:13.884191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d8641-5394-4b3a-b558-8a0fabdf4f7c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:13.884500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d8641-5394-4b3a-b558-8a0fabdf4f7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:13.885687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d8641-5394-4b3a-b558-8a0fabdf4f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:13.887130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d8641-5394-4b3a-b558-8a0fabdf4f7c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:13.889134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 508/1013] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:13.895211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d8641-5394-4b3a-b558-8a0fabdf4f7c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:13.915427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fcfae5-6ac6-48ef-a3cb-e384b5dc02b9 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:13.950930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c5828-2294-482e-9957-268358562d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:13.951596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c5828-2294-482e-9957-268358562d1d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:13.951884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c5828-2294-482e-9957-268358562d1d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:13.952159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c5828-2294-482e-9957-268358562d1d None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:13.953002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c5828-2294-482e-9957-268358562d1d None admin] RBAC: Adding query filter params (name=compute) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:13.953418 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c5828-2294-482e-9957-268358562d1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:13.955329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c5828-2294-482e-9957-268358562d1d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:13.964695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 506/1014] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:38:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:13.966359 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c5828-2294-482e-9957-268358562d1d None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:13.998441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d8641-5394-4b3a-b558-8a0fabdf4f7c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:14.018698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:14.036652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2d2e-5467-4940-be8e-240da2f67ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:14.037331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2d2e-5467-4940-be8e-240da2f67ed8 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:14.037619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2d2e-5467-4940-be8e-240da2f67ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:14.037900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2d2e-5467-4940-be8e-240da2f67ed8 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:14.038613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc2d2e-5467-4940-be8e-240da2f67ed8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:14.040041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc2d2e-5467-4940-be8e-240da2f67ed8 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:14.071130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c5828-2294-482e-9957-268358562d1d None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:14.104905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3d844-6668-4a6f-9a55-2ff2749c33cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:14.105574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3d844-6668-4a6f-9a55-2ff2749c33cd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:14.105865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3d844-6668-4a6f-9a55-2ff2749c33cd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:14.106146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3d844-6668-4a6f-9a55-2ff2749c33cd None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:14.106905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3d844-6668-4a6f-9a55-2ff2749c33cd None admin] RBAC: Adding query filter params (type=compute) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:14.107212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3d844-6668-4a6f-9a55-2ff2749c33cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:14.108634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3d844-6668-4a6f-9a55-2ff2749c33cd None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:14.109866 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:14.113974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 507/1015] 10.222.0.23 () {56 vars in 1112 bytes} [Mon May 23 15:38:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:14.119513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3d844-6668-4a6f-9a55-2ff2749c33cd None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:14.144311 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 509/1016] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:38:13 2022] POST /identity/v3/projects => generated 311 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:38:14.243858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3d844-6668-4a6f-9a55-2ff2749c33cd None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:14.283442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4bae6-d1d2-440c-b6d3-1989fbe6698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:14.284137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4bae6-d1d2-440c-b6d3-1989fbe6698e None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:14.287310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4bae6-d1d2-440c-b6d3-1989fbe6698e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:14.287310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4bae6-d1d2-440c-b6d3-1989fbe6698e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:14.287310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4bae6-d1d2-440c-b6d3-1989fbe6698e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:14.287310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4bae6-d1d2-440c-b6d3-1989fbe6698e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:14.338210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 508/1017] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 23 15:38:14 2022] POST /identity/v3/endpoints => generated 334 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:38:14.421565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616f521-0e20-4070-985e-89d94ed4dc42 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:14.421755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616f521-0e20-4070-985e-89d94ed4dc42 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:14.422049 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616f521-0e20-4070-985e-89d94ed4dc42 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:14.423483 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 510/1018] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:14.429526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecb0cb-5149-4f5b-b7fa-7d0d83df073b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:14.429836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecb0cb-5149-4f5b-b7fa-7d0d83df073b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:14.430098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecb0cb-5149-4f5b-b7fa-7d0d83df073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:14.448341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-22ecb0cb-5149-4f5b-b7fa-7d0d83df073b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:14.449593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-22ecb0cb-5149-4f5b-b7fa-7d0d83df073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:14.514773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:14.540121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 509/1019] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:14 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:14.564087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616f521-0e20-4070-985e-89d94ed4dc42 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:14.671187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616f521-0e20-4070-985e-89d94ed4dc42 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:14.705678 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c1f2b-fe43-40ba-a17c-f3cde88ba169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:14.706272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c1f2b-fe43-40ba-a17c-f3cde88ba169 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:14.706559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c1f2b-fe43-40ba-a17c-f3cde88ba169 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:14.706835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c1f2b-fe43-40ba-a17c-f3cde88ba169 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:14.707561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c1f2b-fe43-40ba-a17c-f3cde88ba169 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:14.708952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c1f2b-fe43-40ba-a17c-f3cde88ba169 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:14.777900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 511/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:38:14 2022] POST /identity/v3/services => generated 242 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:38:16.413261 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756ee0b-3b26-4fd0-8a68-ec194b5acfd2 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:16.413815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756ee0b-3b26-4fd0-8a68-ec194b5acfd2 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:16.413815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756ee0b-3b26-4fd0-8a68-ec194b5acfd2 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:16.415191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 510/1021] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:16.420681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67004fc-afed-43e2-8445-b5f4e3f1ba5f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:16.421102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67004fc-afed-43e2-8445-b5f4e3f1ba5f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:16.421240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67004fc-afed-43e2-8445-b5f4e3f1ba5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:16.438093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c67004fc-afed-43e2-8445-b5f4e3f1ba5f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:16.439770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c67004fc-afed-43e2-8445-b5f4e3f1ba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:16.465444 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:16.521126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 512/1022] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:16.545422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756ee0b-3b26-4fd0-8a68-ec194b5acfd2 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:16.655984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756ee0b-3b26-4fd0-8a68-ec194b5acfd2 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:16.667489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e80da2-5cca-4286-94f2-9db4403c3cdb None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:16.667775 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e80da2-5cca-4286-94f2-9db4403c3cdb None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:16.668223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e80da2-5cca-4286-94f2-9db4403c3cdb None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:16.669867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 513/1023] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:16.675087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d22ba9-d6d7-4145-8370-0a49b88676d0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:16.675377 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d22ba9-d6d7-4145-8370-0a49b88676d0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:16.675658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d22ba9-d6d7-4145-8370-0a49b88676d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:16.693879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e4d22ba9-d6d7-4145-8370-0a49b88676d0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:16.695064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e4d22ba9-d6d7-4145-8370-0a49b88676d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:16.696400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e83c2d-a933-4239-8841-a4b2e67da7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:16.697481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83c2d-a933-4239-8841-a4b2e67da7c0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:16.698629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83c2d-a933-4239-8841-a4b2e67da7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:16.698930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e83c2d-a933-4239-8841-a4b2e67da7c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:16.700128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e83c2d-a933-4239-8841-a4b2e67da7c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:16.701601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e83c2d-a933-4239-8841-a4b2e67da7c0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:16.708768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 511/1024] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:16.710248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e83c2d-a933-4239-8841-a4b2e67da7c0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:16.761404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:16.767215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 514/1025] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:16.790720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d22ba9-d6d7-4145-8370-0a49b88676d0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:16.817060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e83c2d-a933-4239-8841-a4b2e67da7c0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:16.852709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e59c7-4214-48df-a1d6-1197b3b42c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:16.853375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e59c7-4214-48df-a1d6-1197b3b42c3e None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:16.853660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e59c7-4214-48df-a1d6-1197b3b42c3e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:16.853939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e59c7-4214-48df-a1d6-1197b3b42c3e None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:16.854642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e59c7-4214-48df-a1d6-1197b3b42c3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:16.856064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e59c7-4214-48df-a1d6-1197b3b42c3e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:16.881800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:16.896315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d22ba9-d6d7-4145-8370-0a49b88676d0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:16.928900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1fdbc-791e-4e76-8ee9-4c3bea059ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:16.929721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1fdbc-791e-4e76-8ee9-4c3bea059ff0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:16.930004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1fdbc-791e-4e76-8ee9-4c3bea059ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:16.930277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1fdbc-791e-4e76-8ee9-4c3bea059ff0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:16.940609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1fdbc-791e-4e76-8ee9-4c3bea059ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-b4b1fdbc-791e-4e76-8ee9-4c3bea059ff0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:16.948716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:16.949724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:16.951096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 515/1026] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 23 15:38:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:16.955486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 512/1027] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:38:16 2022] POST /identity/v3/users => generated 285 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:38:16.955771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1fdbc-791e-4e76-8ee9-4c3bea059ff0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:16.988727 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2501c24-28e1-44cb-9c32-70b819a6b6f6 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:16.989107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2501c24-28e1-44cb-9c32-70b819a6b6f6 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:16.989293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2501c24-28e1-44cb-9c32-70b819a6b6f6 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:16.990875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 513/1028] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:16.996602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d7c56-819f-49eb-b0d7-f7a3cadb1ee3 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:16.997169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d7c56-819f-49eb-b0d7-f7a3cadb1ee3 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:16.997457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d7c56-819f-49eb-b0d7-f7a3cadb1ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.017505 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-670d7c56-819f-49eb-b0d7-f7a3cadb1ee3 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:17.018742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-670d7c56-819f-49eb-b0d7-f7a3cadb1ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:17.079478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1fdbc-791e-4e76-8ee9-4c3bea059ff0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:17.097502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 514/1029] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:17.121309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d7c56-819f-49eb-b0d7-f7a3cadb1ee3 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:17.122979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cface27-977b-4bd5-9b2b-186623f8d4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:17.123667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cface27-977b-4bd5-9b2b-186623f8d4a6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:17.123949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cface27-977b-4bd5-9b2b-186623f8d4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:17.131079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cface27-977b-4bd5-9b2b-186623f8d4a6 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.131079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cface27-977b-4bd5-9b2b-186623f8d4a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:17.131079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cface27-977b-4bd5-9b2b-186623f8d4a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:17.131079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cface27-977b-4bd5-9b2b-186623f8d4a6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:17.136183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 516/1030] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 23 15:38:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:17.143016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cface27-977b-4bd5-9b2b-186623f8d4a6 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:17.155221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:17.158214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:17.235924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d7c56-819f-49eb-b0d7-f7a3cadb1ee3 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:17.259057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cface27-977b-4bd5-9b2b-186623f8d4a6 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:17.273831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eb0d5-8a76-4785-b84e-08a9c6b53e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:17.274713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eb0d5-8a76-4785-b84e-08a9c6b53e0a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:17.275074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eb0d5-8a76-4785-b84e-08a9c6b53e0a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:17.275496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eb0d5-8a76-4785-b84e-08a9c6b53e0a None admin] PATH_INFO: `/v3/services/image` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.276081 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2eb0d5-8a76-4785-b84e-08a9c6b53e0a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:17.277506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2eb0d5-8a76-4785-b84e-08a9c6b53e0a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-1c2eb0d5-8a76-4785-b84e-08a9c6b53e0a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:17.293658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:17.301083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:17.301939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 515/1031] 10.222.0.23 () {56 vars in 1105 bytes} [Mon May 23 15:38:17 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:17.302754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266761f5-a664-4d95-9b4c-9bbafb27a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:17.302754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266761f5-a664-4d95-9b4c-9bbafb27a16c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:17.302754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266761f5-a664-4d95-9b4c-9bbafb27a16c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:17.302958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266761f5-a664-4d95-9b4c-9bbafb27a16c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.304302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eb0d5-8a76-4785-b84e-08a9c6b53e0a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:17.316304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266761f5-a664-4d95-9b4c-9bbafb27a16c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:17.317831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266761f5-a664-4d95-9b4c-9bbafb27a16c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-266761f5-a664-4d95-9b4c-9bbafb27a16c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:17.335743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:17.336959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:17.337885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:17.337885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:17.337885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:17.337885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:17.337885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 517/1032] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:17.343098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266761f5-a664-4d95-9b4c-9bbafb27a16c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:17.409800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:17.410971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:17.414562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eb0d5-8a76-4785-b84e-08a9c6b53e0a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:17.450630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e6c86-15e4-4fc0-b78f-7906fa63a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:17.451415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e6c86-15e4-4fc0-b78f-7906fa63a78d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:17.451602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e6c86-15e4-4fc0-b78f-7906fa63a78d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:17.451845 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e6c86-15e4-4fc0-b78f-7906fa63a78d None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.452616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e6c86-15e4-4fc0-b78f-7906fa63a78d None admin] RBAC: Adding query filter params (name=image) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:17.452923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e6c86-15e4-4fc0-b78f-7906fa63a78d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:17.454726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266761f5-a664-4d95-9b4c-9bbafb27a16c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:17.455151 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e6c86-15e4-4fc0-b78f-7906fa63a78d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:17.466073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 516/1033] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:17.467045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e6c86-15e4-4fc0-b78f-7906fa63a78d None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:17.495078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfea23-2bdd-4c58-8360-3586a5114c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:17.495745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfea23-2bdd-4c58-8360-3586a5114c08 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:17.496031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfea23-2bdd-4c58-8360-3586a5114c08 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:17.496309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfea23-2bdd-4c58-8360-3586a5114c08 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.497072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcfea23-2bdd-4c58-8360-3586a5114c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:17.497473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcfea23-2bdd-4c58-8360-3586a5114c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:17.498916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcfea23-2bdd-4c58-8360-3586a5114c08 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:17.511648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 518/1034] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:17.517915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfea23-2bdd-4c58-8360-3586a5114c08 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:17.574396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e6c86-15e4-4fc0-b78f-7906fa63a78d None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:17.613041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac260d-a102-4dcb-a30e-1f449aa21332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:17.613682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac260d-a102-4dcb-a30e-1f449aa21332 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:17.613915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac260d-a102-4dcb-a30e-1f449aa21332 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:17.614189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac260d-a102-4dcb-a30e-1f449aa21332 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.614921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac260d-a102-4dcb-a30e-1f449aa21332 None admin] RBAC: Adding query filter params (type=image) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:17.615228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac260d-a102-4dcb-a30e-1f449aa21332 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:17.616895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac260d-a102-4dcb-a30e-1f449aa21332 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:17.627780 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 517/1035] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:17.627926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfea23-2bdd-4c58-8360-3586a5114c08 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:17.630529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac260d-a102-4dcb-a30e-1f449aa21332 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:17.648654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:17.659367 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:17.663062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f375b-3eca-4808-bddc-c85b042b7cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:17.663866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f375b-3eca-4808-bddc-c85b042b7cc6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:17.664175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f375b-3eca-4808-bddc-c85b042b7cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:17.664520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f375b-3eca-4808-bddc-c85b042b7cc6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.671752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f375b-3eca-4808-bddc-c85b042b7cc6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:17.673781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f375b-3eca-4808-bddc-c85b042b7cc6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-739f375b-3eca-4808-bddc-c85b042b7cc6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:38:17.688159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:17.689701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:17.695559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 519/1036] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 23 15:38:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:17.696162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f375b-3eca-4808-bddc-c85b042b7cc6 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:17.741635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac260d-a102-4dcb-a30e-1f449aa21332 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:17.777157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a229d1-85c3-4638-9aff-8f68d5896d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:17.777820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a229d1-85c3-4638-9aff-8f68d5896d99 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:17.778104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a229d1-85c3-4638-9aff-8f68d5896d99 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:17.778381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a229d1-85c3-4638-9aff-8f68d5896d99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.779163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a229d1-85c3-4638-9aff-8f68d5896d99 None admin] RBAC: Adding query filter params (interface=public, service_id=807062f70269479eb06df684482ad71d, region_id=RegionOne) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:17.779566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a229d1-85c3-4638-9aff-8f68d5896d99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:17.780985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a229d1-85c3-4638-9aff-8f68d5896d99 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:17.788399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 518/1037] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 23 15:38:17 2022] GET /identity/v3/endpoints?service_id=807062f70269479eb06df684482ad71d&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:17.791407 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a229d1-85c3-4638-9aff-8f68d5896d99 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:17.806446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f375b-3eca-4808-bddc-c85b042b7cc6 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:17.843454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60423c1f-f708-4851-aae1-33381368efdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:17.844129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60423c1f-f708-4851-aae1-33381368efdd None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:17.844437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60423c1f-f708-4851-aae1-33381368efdd None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:17.844746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60423c1f-f708-4851-aae1-33381368efdd None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.845585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60423c1f-f708-4851-aae1-33381368efdd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:17.845899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60423c1f-f708-4851-aae1-33381368efdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:17.847315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60423c1f-f708-4851-aae1-33381368efdd None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:17.868548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 520/1038] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 23 15:38:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:17.873895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60423c1f-f708-4851-aae1-33381368efdd None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:17.887448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:17.899390 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a229d1-85c3-4638-9aff-8f68d5896d99 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:17.913546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:17.939860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb63037-4368-4c8f-bb36-4b553ebde765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:17.940557 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb63037-4368-4c8f-bb36-4b553ebde765 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:17.940880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb63037-4368-4c8f-bb36-4b553ebde765 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:17.941228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb63037-4368-4c8f-bb36-4b553ebde765 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:17.942160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb63037-4368-4c8f-bb36-4b553ebde765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:17.943302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb63037-4368-4c8f-bb36-4b553ebde765 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:17.960391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 519/1039] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 23 15:38:17 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:38:17.990074 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60423c1f-f708-4851-aae1-33381368efdd None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:18.027464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f7b7a-aee6-4c7e-b14f-c8221068b839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:18.028136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f7b7a-aee6-4c7e-b14f-c8221068b839 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:18.028462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f7b7a-aee6-4c7e-b14f-c8221068b839 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:18.028739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f7b7a-aee6-4c7e-b14f-c8221068b839 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:18.101019 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f7b7a-aee6-4c7e-b14f-c8221068b839 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:18.102470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f7b7a-aee6-4c7e-b14f-c8221068b839 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:18.148330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:18.209806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 521/1040] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:17 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:38:18.215354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb63037-4368-4c8f-bb36-4b553ebde765 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:18.323862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb63037-4368-4c8f-bb36-4b553ebde765 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:18.357937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29884-f845-47b4-97f3-6d991b3827be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:18.358754 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29884-f845-47b4-97f3-6d991b3827be None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:18.359038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29884-f845-47b4-97f3-6d991b3827be None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:18.359313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29884-f845-47b4-97f3-6d991b3827be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:18.369246 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd29884-f845-47b4-97f3-6d991b3827be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:18.370676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd29884-f845-47b4-97f3-6d991b3827be None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-fbd29884-f845-47b4-97f3-6d991b3827be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:18.387460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:18.388643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:18.389522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:18.389522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:18.389522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:18.389522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:18.391144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 520/1041] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:18.395552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f7b7a-aee6-4c7e-b14f-c8221068b839 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:18.497781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f7b7a-aee6-4c7e-b14f-c8221068b839 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:18.535022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f8c8d-f3e1-46be-863a-a1bdbceff6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:18.536053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f8c8d-f3e1-46be-863a-a1bdbceff6a0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:18.536583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f8c8d-f3e1-46be-863a-a1bdbceff6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:18.537064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f8c8d-f3e1-46be-863a-a1bdbceff6a0 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:18.537983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f8c8d-f3e1-46be-863a-a1bdbceff6a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:18.538491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f8c8d-f3e1-46be-863a-a1bdbceff6a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:18.540104 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f8c8d-f3e1-46be-863a-a1bdbceff6a0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:18.547448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:18.556819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 522/1042] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:18.558394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29884-f845-47b4-97f3-6d991b3827be None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:18.568047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:18.659899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29884-f845-47b4-97f3-6d991b3827be None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:18.693259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd59a2-a779-4514-8335-5498f498bae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:18.694053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd59a2-a779-4514-8335-5498f498bae1 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:18.694333 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd59a2-a779-4514-8335-5498f498bae1 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:18.694607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd59a2-a779-4514-8335-5498f498bae1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:18.701240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd59a2-a779-4514-8335-5498f498bae1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:18.702680 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd59a2-a779-4514-8335-5498f498bae1 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-5fbd59a2-a779-4514-8335-5498f498bae1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:38:18.717126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:18.718029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:18.718966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:38:18.718966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:38:18.718966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:38:18.718966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:38:18.718966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:38:18.718966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:38:18.718966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:18.718966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:18.720685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 521/1043] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:18.724253 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f8c8d-f3e1-46be-863a-a1bdbceff6a0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:18.823963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f8c8d-f3e1-46be-863a-a1bdbceff6a0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:18.856631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee3fbd-8e0e-4517-9b48-de0380b9c141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:18.857289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee3fbd-8e0e-4517-9b48-de0380b9c141 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:18.857571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee3fbd-8e0e-4517-9b48-de0380b9c141 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:18.857846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee3fbd-8e0e-4517-9b48-de0380b9c141 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:18.858625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee3fbd-8e0e-4517-9b48-de0380b9c141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:18.858934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee3fbd-8e0e-4517-9b48-de0380b9c141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:18.860372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee3fbd-8e0e-4517-9b48-de0380b9c141 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:18.876404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 523/1044] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:38:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:18.879025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd59a2-a779-4514-8335-5498f498bae1 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:18.946394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:18.980666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd59a2-a779-4514-8335-5498f498bae1 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:19.015395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1efc97-bf6f-4cba-8f65-be84964a4386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:19.016071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1efc97-bf6f-4cba-8f65-be84964a4386 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:19.016374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1efc97-bf6f-4cba-8f65-be84964a4386 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:19.017222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1efc97-bf6f-4cba-8f65-be84964a4386 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:19.018062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1efc97-bf6f-4cba-8f65-be84964a4386 None admin] RBAC: Adding query filter params (role.id=8ad5ca9a5ad04e7fbb93d1330cf468e2, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=b9129f0b362a4945a0eca7cc33571abd) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:19.018370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1efc97-bf6f-4cba-8f65-be84964a4386 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:19.019876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1efc97-bf6f-4cba-8f65-be84964a4386 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:19.032683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 522/1045] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:38:18 2022] GET /identity/v3/role_assignments?user.id=b9129f0b362a4945a0eca7cc33571abd&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=8ad5ca9a5ad04e7fbb93d1330cf468e2 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:19.225965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee05e-8263-4134-9a9b-012309c59ac1 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:19.226623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee05e-8263-4134-9a9b-012309c59ac1 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:19.227134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee05e-8263-4134-9a9b-012309c59ac1 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:19.229056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 524/1046] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:19.233984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988082e-88c7-4ad3-815f-a3987e3b033c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:19.234279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988082e-88c7-4ad3-815f-a3987e3b033c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:19.234559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988082e-88c7-4ad3-815f-a3987e3b033c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:19.252270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2988082e-88c7-4ad3-815f-a3987e3b033c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:19.253491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2988082e-88c7-4ad3-815f-a3987e3b033c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:19.329635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 523/1047] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:19.352234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee05e-8263-4134-9a9b-012309c59ac1 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:19.414150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:19.455848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee05e-8263-4134-9a9b-012309c59ac1 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:19.490450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe7228-c064-4daf-a924-c6a3ac5983bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:19.491167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe7228-c064-4daf-a924-c6a3ac5983bb None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:19.491448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe7228-c064-4daf-a924-c6a3ac5983bb None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:19.491724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe7228-c064-4daf-a924-c6a3ac5983bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:19.501861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe7228-c064-4daf-a924-c6a3ac5983bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-b4fe7228-c064-4daf-a924-c6a3ac5983bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:19.509443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:19.510383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:19.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 525/1048] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:38:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:19.517138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988082e-88c7-4ad3-815f-a3987e3b033c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:19.575046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:19.617069 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988082e-88c7-4ad3-815f-a3987e3b033c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:19.649867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9d0d4-568e-4f8a-b861-dfea18dc5852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:19.650545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9d0d4-568e-4f8a-b861-dfea18dc5852 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:19.650827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9d0d4-568e-4f8a-b861-dfea18dc5852 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:19.651104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9d0d4-568e-4f8a-b861-dfea18dc5852 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:19.651848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9d0d4-568e-4f8a-b861-dfea18dc5852 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:19.652159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9d0d4-568e-4f8a-b861-dfea18dc5852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:19.653580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9d0d4-568e-4f8a-b861-dfea18dc5852 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:19.666858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 524/1049] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:38:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:19.668227 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe7228-c064-4daf-a924-c6a3ac5983bb None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:19.769039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe7228-c064-4daf-a924-c6a3ac5983bb None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:19.786451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:19.803136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdde3c6-e2fb-4299-bc20-e72b4bcf0884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:19.804011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdde3c6-e2fb-4299-bc20-e72b4bcf0884 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:19.804298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdde3c6-e2fb-4299-bc20-e72b4bcf0884 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:19.804602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdde3c6-e2fb-4299-bc20-e72b4bcf0884 None admin] PATH_INFO: `/v3/users/5c96ffe1dd444559936512197dd76902` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:19.818689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdde3c6-e2fb-4299-bc20-e72b4bcf0884 None admin] RBAC: Authorizing `identity:get_user(user_id=5c96ffe1dd444559936512197dd76902)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:19.820128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdde3c6-e2fb-4299-bc20-e72b4bcf0884 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:19.826692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 526/1050] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:19 2022] GET /identity/v3/users/5c96ffe1dd444559936512197dd76902 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:19.827543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9d0d4-568e-4f8a-b861-dfea18dc5852 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:19.926367 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9d0d4-568e-4f8a-b861-dfea18dc5852 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:19.947877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:19.959886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180bad3-dc45-4c9d-8615-5cea5995d0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:19.960572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180bad3-dc45-4c9d-8615-5cea5995d0df None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:19.960859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180bad3-dc45-4c9d-8615-5cea5995d0df None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:19.961932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180bad3-dc45-4c9d-8615-5cea5995d0df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:20.034580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180bad3-dc45-4c9d-8615-5cea5995d0df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:20.035983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180bad3-dc45-4c9d-8615-5cea5995d0df None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:20.139077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 525/1051] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:19 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:38:20.140298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdde3c6-e2fb-4299-bc20-e72b4bcf0884 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:20.183641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed701-cc31-41ee-a285-f12ab7181941 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:20.183940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed701-cc31-41ee-a285-f12ab7181941 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:20.184226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed701-cc31-41ee-a285-f12ab7181941 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:20.185648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 526/1052] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:20.191106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48aeaf-7089-4475-ad75-d0517f2efbf9 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:20.191398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48aeaf-7089-4475-ad75-d0517f2efbf9 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:20.191677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48aeaf-7089-4475-ad75-d0517f2efbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:20.209732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-bf48aeaf-7089-4475-ad75-d0517f2efbf9 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:20.210924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bf48aeaf-7089-4475-ad75-d0517f2efbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:20.228904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:20.245408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdde3c6-e2fb-4299-bc20-e72b4bcf0884 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:20.284133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 527/1053] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:20.298462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88adcf-0d14-4947-8a36-a223da3a5943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:20.299406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88adcf-0d14-4947-8a36-a223da3a5943 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:20.299760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88adcf-0d14-4947-8a36-a223da3a5943 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:20.300108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88adcf-0d14-4947-8a36-a223da3a5943 None admin] PATH_INFO: `/v3/projects/71195db0f36c4a7da07582dad1ab5284` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:20.301410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88adcf-0d14-4947-8a36-a223da3a5943 None admin] RBAC: Authorizing `identity:get_project(project_id=71195db0f36c4a7da07582dad1ab5284)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:20.302833 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88adcf-0d14-4947-8a36-a223da3a5943 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:20.309523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 527/1054] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:38:20 2022] GET /identity/v3/projects/71195db0f36c4a7da07582dad1ab5284 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:20.310349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48aeaf-7089-4475-ad75-d0517f2efbf9 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:20.310677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88adcf-0d14-4947-8a36-a223da3a5943 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:20.360896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:20.415269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88adcf-0d14-4947-8a36-a223da3a5943 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:20.422235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48aeaf-7089-4475-ad75-d0517f2efbf9 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:20.451631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676be228-992d-478e-9979-6b0370e61a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:20.452292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be228-992d-478e-9979-6b0370e61a4c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:20.452612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be228-992d-478e-9979-6b0370e61a4c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:20.452893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be228-992d-478e-9979-6b0370e61a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:20.453768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676be228-992d-478e-9979-6b0370e61a4c None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=71195db0f36c4a7da07582dad1ab5284, user.id=5c96ffe1dd444559936512197dd76902) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:20.454087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676be228-992d-478e-9979-6b0370e61a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:20.456022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676be228-992d-478e-9979-6b0370e61a4c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:20.457757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2afba9-fe3a-4d32-bfa1-b93de5041d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:20.458545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2afba9-fe3a-4d32-bfa1-b93de5041d18 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:20.458835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2afba9-fe3a-4d32-bfa1-b93de5041d18 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:20.459116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2afba9-fe3a-4d32-bfa1-b93de5041d18 None admin] PATH_INFO: `/v3/services/image` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:20.459760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2afba9-fe3a-4d32-bfa1-b93de5041d18 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:20.461199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2afba9-fe3a-4d32-bfa1-b93de5041d18 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:20.472654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 528/1055] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:38:20 2022] GET /identity/v3/role_assignments?user.id=5c96ffe1dd444559936512197dd76902&scope.project.id=71195db0f36c4a7da07582dad1ab5284&role.id=04c11dea81274a69974ad5c639f2c60b => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ec2afba9-fe3a-4d32-bfa1-b93de5041d18 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:20.483906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:20.485187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:20.486107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 528/1056] 10.222.0.23 () {56 vars in 1105 bytes} [Mon May 23 15:38:20 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:20.492029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676be228-992d-478e-9979-6b0370e61a4c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:20.601381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676be228-992d-478e-9979-6b0370e61a4c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:20.612400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:20.639955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9166c-61f2-4050-8f15-9598923bfc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:20.640708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9166c-61f2-4050-8f15-9598923bfc1e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:20.641005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9166c-61f2-4050-8f15-9598923bfc1e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:20.641286 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9166c-61f2-4050-8f15-9598923bfc1e None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:20.642038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9166c-61f2-4050-8f15-9598923bfc1e None admin] RBAC: Adding query filter params (name=image) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:20.642351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9166c-61f2-4050-8f15-9598923bfc1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:20.643829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9166c-61f2-4050-8f15-9598923bfc1e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:20.649463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 529/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:20.654810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2afba9-fe3a-4d32-bfa1-b93de5041d18 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:20.674883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:20.759505 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2afba9-fe3a-4d32-bfa1-b93de5041d18 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:20.794922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2896521-334d-45c6-8332-35dd4fc6e7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:20.795606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2896521-334d-45c6-8332-35dd4fc6e7dc None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:20.795891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2896521-334d-45c6-8332-35dd4fc6e7dc None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:20.796170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2896521-334d-45c6-8332-35dd4fc6e7dc None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:20.796926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2896521-334d-45c6-8332-35dd4fc6e7dc None admin] RBAC: Adding query filter params (type=image) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:20.797239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2896521-334d-45c6-8332-35dd4fc6e7dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:20.798646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2896521-334d-45c6-8332-35dd4fc6e7dc None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:20.808289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 529/1058] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:20.809305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9166c-61f2-4050-8f15-9598923bfc1e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:20.909996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9166c-61f2-4050-8f15-9598923bfc1e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:20.944049 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e48268-1716-41a5-ba21-1831e3c914f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:20.944745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e48268-1716-41a5-ba21-1831e3c914f5 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:20.945032 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e48268-1716-41a5-ba21-1831e3c914f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:20.945312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e48268-1716-41a5-ba21-1831e3c914f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:20.946061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e48268-1716-41a5-ba21-1831e3c914f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:20.947427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e48268-1716-41a5-ba21-1831e3c914f5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:21.035051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 530/1059] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 23 15:38:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:38:21.242954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09007a25-39ac-45d4-9342-925c5cdb9363 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:21.243166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09007a25-39ac-45d4-9342-925c5cdb9363 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:21.243445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09007a25-39ac-45d4-9342-925c5cdb9363 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:21.244877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 530/1060] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:21.250406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb0415-ac2f-4252-9867-ba6a6a35f098 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:21.250705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb0415-ac2f-4252-9867-ba6a6a35f098 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:21.251536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb0415-ac2f-4252-9867-ba6a6a35f098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:21.269891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-61eb0415-ac2f-4252-9867-ba6a6a35f098 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:21.271088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-61eb0415-ac2f-4252-9867-ba6a6a35f098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:21.294950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:21.346434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 531/1061] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:21.369135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09007a25-39ac-45d4-9342-925c5cdb9363 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:21.451095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:21.470987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09007a25-39ac-45d4-9342-925c5cdb9363 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:21.504334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea78733-e25d-4ef5-950f-004e24d12c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:21.505176 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78733-e25d-4ef5-950f-004e24d12c77 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:21.505466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78733-e25d-4ef5-950f-004e24d12c77 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:21.505746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea78733-e25d-4ef5-950f-004e24d12c77 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:21.516642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea78733-e25d-4ef5-950f-004e24d12c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-0ea78733-e25d-4ef5-950f-004e24d12c77 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:21.524830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:21.525735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:38:21.526606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:21.528435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 531/1062] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 23 15:38:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:21.532034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb0415-ac2f-4252-9867-ba6a6a35f098 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:21.639943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb0415-ac2f-4252-9867-ba6a6a35f098 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:21.674735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f800e61-0fc3-4c72-aad1-8b9d91145cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:21.675411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f800e61-0fc3-4c72-aad1-8b9d91145cfc None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:21.675695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f800e61-0fc3-4c72-aad1-8b9d91145cfc None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:21.675976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f800e61-0fc3-4c72-aad1-8b9d91145cfc None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:21.676776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f800e61-0fc3-4c72-aad1-8b9d91145cfc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:21.677089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f800e61-0fc3-4c72-aad1-8b9d91145cfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:21.678448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f800e61-0fc3-4c72-aad1-8b9d91145cfc None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:21.686694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 532/1063] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 23 15:38:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:21.692446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea78733-e25d-4ef5-950f-004e24d12c77 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:21.791973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea78733-e25d-4ef5-950f-004e24d12c77 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:21.825623 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675944-832d-42a1-9d98-ca1da7c4107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:21.826417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675944-832d-42a1-9d98-ca1da7c4107f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:21.826702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675944-832d-42a1-9d98-ca1da7c4107f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:21.826976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675944-832d-42a1-9d98-ca1da7c4107f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:21.835617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:21.838664 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca675944-832d-42a1-9d98-ca1da7c4107f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:21.840112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca675944-832d-42a1-9d98-ca1da7c4107f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ca675944-832d-42a1-9d98-ca1da7c4107f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:21.857360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:21.858276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:21.864796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:21.864796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:21.864796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:21.864796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:21.864796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 532/1064] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:21.865088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f800e61-0fc3-4c72-aad1-8b9d91145cfc None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:21.884601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:21.965898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f800e61-0fc3-4c72-aad1-8b9d91145cfc None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:22.000176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a04e2b-a99c-446a-8a6f-fb83308a4865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:22.000870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a04e2b-a99c-446a-8a6f-fb83308a4865 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.001183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a04e2b-a99c-446a-8a6f-fb83308a4865 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.001462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a04e2b-a99c-446a-8a6f-fb83308a4865 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.002189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a04e2b-a99c-446a-8a6f-fb83308a4865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:22.002499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a04e2b-a99c-446a-8a6f-fb83308a4865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:22.003865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a04e2b-a99c-446a-8a6f-fb83308a4865 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:22.020162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 533/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:22.022048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675944-832d-42a1-9d98-ca1da7c4107f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:22.122940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675944-832d-42a1-9d98-ca1da7c4107f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:22.158298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0c2ef-17da-4d32-b852-84e40a922a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:22.159090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0c2ef-17da-4d32-b852-84e40a922a4e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.159376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0c2ef-17da-4d32-b852-84e40a922a4e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.159652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0c2ef-17da-4d32-b852-84e40a922a4e None admin] PATH_INFO: `/v3/users/nova` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.168059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0c2ef-17da-4d32-b852-84e40a922a4e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:22.169606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0c2ef-17da-4d32-b852-84e40a922a4e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-47a0c2ef-17da-4d32-b852-84e40a922a4e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:38:22.184764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:22.185773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:22.191412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 533/1066] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 23 15:38:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:22.192039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a04e2b-a99c-446a-8a6f-fb83308a4865 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:22.271788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:22.299395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a04e2b-a99c-446a-8a6f-fb83308a4865 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:22.333273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee169f-e4c1-4781-9f0c-ea85ad7a4b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:22.333947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee169f-e4c1-4781-9f0c-ea85ad7a4b7f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.334231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee169f-e4c1-4781-9f0c-ea85ad7a4b7f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.334504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee169f-e4c1-4781-9f0c-ea85ad7a4b7f None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.335337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee169f-e4c1-4781-9f0c-ea85ad7a4b7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:22.335643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee169f-e4c1-4781-9f0c-ea85ad7a4b7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:22.337091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee169f-e4c1-4781-9f0c-ea85ad7a4b7f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:22.358788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 534/1067] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 23 15:38:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:22.360575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0c2ef-17da-4d32-b852-84e40a922a4e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:22.413911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:22.461462 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0c2ef-17da-4d32-b852-84e40a922a4e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:22.499106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdabe7-0d14-4dfd-bdd7-4efd2ac7bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:22.499908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdabe7-0d14-4dfd-bdd7-4efd2ac7bf28 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.500192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdabe7-0d14-4dfd-bdd7-4efd2ac7bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.500545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdabe7-0d14-4dfd-bdd7-4efd2ac7bf28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.511976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdabe7-0d14-4dfd-bdd7-4efd2ac7bf28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:22.513723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdabe7-0d14-4dfd-bdd7-4efd2ac7bf28 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-7ffdabe7-0d14-4dfd-bdd7-4efd2ac7bf28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:22.531023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:22.531989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:22.537457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:22.537457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:22.537457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:22.537457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:22.537457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 534/1068] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:22.539304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee169f-e4c1-4781-9f0c-ea85ad7a4b7f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:22.642229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee169f-e4c1-4781-9f0c-ea85ad7a4b7f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:22.660525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1151bec-4643-4cad-9c6e-b4ff55ddbc83 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.660810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1151bec-4643-4cad-9c6e-b4ff55ddbc83 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.661085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1151bec-4643-4cad-9c6e-b4ff55ddbc83 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.662740 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 535/1069] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:22.667509 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675fd4f-b76a-4ab2-ab72-a5e523f44514 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.667730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675fd4f-b76a-4ab2-ab72-a5e523f44514 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.667999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675fd4f-b76a-4ab2-ab72-a5e523f44514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.683392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac2f87-8078-4f2a-a593-7bf526b3c1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:22.684058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac2f87-8078-4f2a-a593-7bf526b3c1fa None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.684342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac2f87-8078-4f2a-a593-7bf526b3c1fa None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.684652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac2f87-8078-4f2a-a593-7bf526b3c1fa None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.685383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac2f87-8078-4f2a-a593-7bf526b3c1fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:22.685692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac2f87-8078-4f2a-a593-7bf526b3c1fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:22.687037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac2f87-8078-4f2a-a593-7bf526b3c1fa None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:22.689158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:22.691130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-f675fd4f-b76a-4ab2-ab72-a5e523f44514 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:22.692309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f675fd4f-b76a-4ab2-ab72-a5e523f44514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:22.700350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 535/1070] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:22.706518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac2f87-8078-4f2a-a593-7bf526b3c1fa None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:22.777405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 536/1071] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:22.796115 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675fd4f-b76a-4ab2-ab72-a5e523f44514 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:22.823757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac2f87-8078-4f2a-a593-7bf526b3c1fa None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:22.830572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:22.866691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf36768-d132-49b9-87a7-80294110d654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:22.868089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf36768-d132-49b9-87a7-80294110d654 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.868463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf36768-d132-49b9-87a7-80294110d654 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.868744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf36768-d132-49b9-87a7-80294110d654 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.877348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf36768-d132-49b9-87a7-80294110d654 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:22.879058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf36768-d132-49b9-87a7-80294110d654 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-ebf36768-d132-49b9-87a7-80294110d654 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:38:22.895021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:22.896120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:22.897272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:38:22.897272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:38:22.897272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:38:22.897272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:38:22.897272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:38:22.897272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:38:22.897272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:22.897272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:22.899089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 536/1072] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:38:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:22.903894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675fd4f-b76a-4ab2-ab72-a5e523f44514 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:22.904994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf36768-d132-49b9-87a7-80294110d654 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:22.941886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fea9ed-4311-4b3b-9f1f-988a8682caf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:22.942622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fea9ed-4311-4b3b-9f1f-988a8682caf4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:22.942908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fea9ed-4311-4b3b-9f1f-988a8682caf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:22.943185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fea9ed-4311-4b3b-9f1f-988a8682caf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:22.955422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fea9ed-4311-4b3b-9f1f-988a8682caf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:22.962643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-b0fea9ed-4311-4b3b-9f1f-988a8682caf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:22.963098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:22.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:22.964926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 537/1073] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:38:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:22.970158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fea9ed-4311-4b3b-9f1f-988a8682caf4 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:23.011468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf36768-d132-49b9-87a7-80294110d654 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:23.045939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fceff-50d5-401b-8d16-72680e3b7bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:23.046619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fceff-50d5-401b-8d16-72680e3b7bfa None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:23.046901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fceff-50d5-401b-8d16-72680e3b7bfa None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:23.047174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fceff-50d5-401b-8d16-72680e3b7bfa None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:23.048551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fceff-50d5-401b-8d16-72680e3b7bfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:23.048863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fceff-50d5-401b-8d16-72680e3b7bfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:23.050300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fceff-50d5-401b-8d16-72680e3b7bfa None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:23.061914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 537/1074] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:38:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:23.068827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fceff-50d5-401b-8d16-72680e3b7bfa None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:23.082205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:23.089844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fea9ed-4311-4b3b-9f1f-988a8682caf4 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:23.137309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98d27-8e64-46af-9f7e-96329daab218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:23.138360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98d27-8e64-46af-9f7e-96329daab218 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:23.138648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98d27-8e64-46af-9f7e-96329daab218 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:23.139190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc98d27-8e64-46af-9f7e-96329daab218 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:23.139806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc98d27-8e64-46af-9f7e-96329daab218 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:23.140133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc98d27-8e64-46af-9f7e-96329daab218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:23.141725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc98d27-8e64-46af-9f7e-96329daab218 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:23.157796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 538/1075] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:38:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:23.160196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98d27-8e64-46af-9f7e-96329daab218 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:23.185756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fceff-50d5-401b-8d16-72680e3b7bfa None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:23.199881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:23.221505 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913014da-c1b3-4430-822a-216681526c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:23.222392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913014da-c1b3-4430-822a-216681526c7f None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:23.222672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913014da-c1b3-4430-822a-216681526c7f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:23.222947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913014da-c1b3-4430-822a-216681526c7f None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/b9129f0b362a4945a0eca7cc33571abd/roles/8ad5ca9a5ad04e7fbb93d1330cf468e2` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:23.225255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913014da-c1b3-4430-822a-216681526c7f None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=b9129f0b362a4945a0eca7cc33571abd, role_id=8ad5ca9a5ad04e7fbb93d1330cf468e2)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:23.226684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913014da-c1b3-4430-822a-216681526c7f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:23.273480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc98d27-8e64-46af-9f7e-96329daab218 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:23.309821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55835a-b2cf-428c-b76b-c93e0071f4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:23.310721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55835a-b2cf-428c-b76b-c93e0071f4fd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:23.311004 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55835a-b2cf-428c-b76b-c93e0071f4fd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:23.311286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55835a-b2cf-428c-b76b-c93e0071f4fd None admin] PATH_INFO: `/v3/users/5c96ffe1dd444559936512197dd76902` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:23.312500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55835a-b2cf-428c-b76b-c93e0071f4fd None admin] RBAC: Authorizing `identity:get_user(user_id=5c96ffe1dd444559936512197dd76902)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:23.313972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55835a-b2cf-428c-b76b-c93e0071f4fd None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:23.315615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 539/1076] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:23 2022] GET /identity/v3/users/5c96ffe1dd444559936512197dd76902 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:23.320506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 538/1077] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:38:23 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/b9129f0b362a4945a0eca7cc33571abd/roles/8ad5ca9a5ad04e7fbb93d1330cf468e2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:38:23.322115 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55835a-b2cf-428c-b76b-c93e0071f4fd None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:23.325016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ee3d8-b0b7-4ec4-ac97-a41298c86eb2 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:23.325594 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ee3d8-b0b7-4ec4-ac97-a41298c86eb2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:23.325777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ee3d8-b0b7-4ec4-ac97-a41298c86eb2 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:23.327343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 539/1078] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:23.333496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d8e73-9eff-4615-9a79-accd073412d4 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:23.333799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d8e73-9eff-4615-9a79-accd073412d4 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:23.334115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d8e73-9eff-4615-9a79-accd073412d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:23.353448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:23.353879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-439d8e73-9eff-4615-9a79-accd073412d4 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:23.355058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-439d8e73-9eff-4615-9a79-accd073412d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:23.433448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 540/1079] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:23.443819 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55835a-b2cf-428c-b76b-c93e0071f4fd None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:23.456516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d8e73-9eff-4615-9a79-accd073412d4 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:23.481593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173f48c-5b16-4cae-8979-30fff77f561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:23.482517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173f48c-5b16-4cae-8979-30fff77f561f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:23.482809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173f48c-5b16-4cae-8979-30fff77f561f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:23.483099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173f48c-5b16-4cae-8979-30fff77f561f None admin] PATH_INFO: `/v3/projects/71195db0f36c4a7da07582dad1ab5284` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:23.484389 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173f48c-5b16-4cae-8979-30fff77f561f None admin] RBAC: Authorizing `identity:get_project(project_id=71195db0f36c4a7da07582dad1ab5284)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:23.486416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173f48c-5b16-4cae-8979-30fff77f561f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:23.493101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 540/1080] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:38:23 2022] GET /identity/v3/projects/71195db0f36c4a7da07582dad1ab5284 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:23.494233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173f48c-5b16-4cae-8979-30fff77f561f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:23.571132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d8e73-9eff-4615-9a79-accd073412d4 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:23.573934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:23.599598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:23.607005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fe4a9-bcf4-472a-b211-ed109845bae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:23.607823 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fe4a9-bcf4-472a-b211-ed109845bae0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:23.608128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173f48c-5b16-4cae-8979-30fff77f561f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:23.608232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fe4a9-bcf4-472a-b211-ed109845bae0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:23.608548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fe4a9-bcf4-472a-b211-ed109845bae0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:23.646238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2251-5b2a-4621-9a57-b021dd93dbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:23.647133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2251-5b2a-4621-9a57-b021dd93dbe0 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:23.647415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2251-5b2a-4621-9a57-b021dd93dbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:23.648088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd2251-5b2a-4621-9a57-b021dd93dbe0 None admin] PATH_INFO: `/v3/projects/71195db0f36c4a7da07582dad1ab5284/users/5c96ffe1dd444559936512197dd76902/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:23.650516 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd2251-5b2a-4621-9a57-b021dd93dbe0 None admin] RBAC: Authorizing `identity:create_grant(project_id=71195db0f36c4a7da07582dad1ab5284, user_id=5c96ffe1dd444559936512197dd76902, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:23.652247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd2251-5b2a-4621-9a57-b021dd93dbe0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:23.677101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773fe4a9-bcf4-472a-b211-ed109845bae0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:23.678553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773fe4a9-bcf4-472a-b211-ed109845bae0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:23.702654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 541/1081] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:38:23 2022] PUT /identity/v3/projects/71195db0f36c4a7da07582dad1ab5284/users/5c96ffe1dd444559936512197dd76902/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:38:23.796005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 541/1082] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:38:23.802612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2251-5b2a-4621-9a57-b021dd93dbe0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:23.911100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd2251-5b2a-4621-9a57-b021dd93dbe0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:23.945431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a932076-27f9-4e41-bde6-66cef891945c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:23.946228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a932076-27f9-4e41-bde6-66cef891945c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:23.946511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a932076-27f9-4e41-bde6-66cef891945c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:23.946786 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a932076-27f9-4e41-bde6-66cef891945c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:23.957231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a932076-27f9-4e41-bde6-66cef891945c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:23.958670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a932076-27f9-4e41-bde6-66cef891945c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-6a932076-27f9-4e41-bde6-66cef891945c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:23.975662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:23.976598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:23.977698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:23.977698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:23.977698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:23.977698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:23.977698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 542/1083] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:23.986552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fe4a9-bcf4-472a-b211-ed109845bae0 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:23.986552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:24.085054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fe4a9-bcf4-472a-b211-ed109845bae0 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:24.118583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d07f3-cdd3-409c-b7fd-58bee09f9a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:24.119284 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d07f3-cdd3-409c-b7fd-58bee09f9a9f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:24.119525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d07f3-cdd3-409c-b7fd-58bee09f9a9f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:24.119798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d07f3-cdd3-409c-b7fd-58bee09f9a9f None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:24.125769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d07f3-cdd3-409c-b7fd-58bee09f9a9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:24.125769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d07f3-cdd3-409c-b7fd-58bee09f9a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:24.125769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d07f3-cdd3-409c-b7fd-58bee09f9a9f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:24.140291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 542/1084] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:25.532722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c0261-cf28-4f52-aa85-c6626e69b81d None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.533237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c0261-cf28-4f52-aa85-c6626e69b81d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.533323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c0261-cf28-4f52-aa85-c6626e69b81d None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.534663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 543/1085] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:25.540037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d94bd-22bd-4c69-83be-9ee71fdf24f7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.540480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d94bd-22bd-4c69-83be-9ee71fdf24f7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.540570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d94bd-22bd-4c69-83be-9ee71fdf24f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.557014 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-369d94bd-22bd-4c69-83be-9ee71fdf24f7 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:25.558183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-369d94bd-22bd-4c69-83be-9ee71fdf24f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:25.621768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:25.631480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 543/1086] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:25.655013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c0261-cf28-4f52-aa85-c6626e69b81d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.658796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:25.759794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c0261-cf28-4f52-aa85-c6626e69b81d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.793932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4026378-528c-4853-8a44-b7a747bdf89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.794672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4026378-528c-4853-8a44-b7a747bdf89a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.794953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4026378-528c-4853-8a44-b7a747bdf89a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.795227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4026378-528c-4853-8a44-b7a747bdf89a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.805642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4026378-528c-4853-8a44-b7a747bdf89a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-b4026378-528c-4853-8a44-b7a747bdf89a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:25.814497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:25.815685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:25.816660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 544/1087] 10.222.0.23 () {56 vars in 1120 bytes} [Mon May 23 15:38:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:25.821584 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d94bd-22bd-4c69-83be-9ee71fdf24f7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.901398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8a48a-039e-4f1d-b965-9bf842d21b4f None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.901691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8a48a-039e-4f1d-b965-9bf842d21b4f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.901975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8a48a-039e-4f1d-b965-9bf842d21b4f None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.903349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 545/1088] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:38:25.908500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbe980-e26a-4d39-b4bd-d0ee8fa151a3 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.908822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbe980-e26a-4d39-b4bd-d0ee8fa151a3 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.909104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbe980-e26a-4d39-b4bd-d0ee8fa151a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.919652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d94bd-22bd-4c69-83be-9ee71fdf24f7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:25.927903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-76bbe980-e26a-4d39-b4bd-d0ee8fa151a3 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:25.929513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-76bbe980-e26a-4d39-b4bd-d0ee8fa151a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:25.954367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a8a4b-a2bf-436c-84cb-cd652813eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:25.955045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a8a4b-a2bf-436c-84cb-cd652813eb4a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:25.955327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a8a4b-a2bf-436c-84cb-cd652813eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:25.955599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a8a4b-a2bf-436c-84cb-cd652813eb4a None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:25.956392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a8a4b-a2bf-436c-84cb-cd652813eb4a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:25.956709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a8a4b-a2bf-436c-84cb-cd652813eb4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:25.958067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a8a4b-a2bf-436c-84cb-cd652813eb4a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:25.965790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 544/1089] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 23 15:38:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:25.971347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a8a4b-a2bf-436c-84cb-cd652813eb4a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:25.988637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:25.996445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 546/1090] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:38:26.020148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbe980-e26a-4d39-b4bd-d0ee8fa151a3 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.027112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:26.072387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a8a4b-a2bf-436c-84cb-cd652813eb4a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.104411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d8ddc-3be1-4d8c-89bd-35a57f0537d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.105205 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d8ddc-3be1-4d8c-89bd-35a57f0537d9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.105487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d8ddc-3be1-4d8c-89bd-35a57f0537d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.105763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d8ddc-3be1-4d8c-89bd-35a57f0537d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.116838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d8ddc-3be1-4d8c-89bd-35a57f0537d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:26.117595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbe980-e26a-4d39-b4bd-d0ee8fa151a3 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.118262 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d8ddc-3be1-4d8c-89bd-35a57f0537d9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-2c4d8ddc-3be1-4d8c-89bd-35a57f0537d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:26.135349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:26.136503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:26.137580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:26.137580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:26.137580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:26.137580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:26.140674 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 545/1091] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:26.142365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d8ddc-3be1-4d8c-89bd-35a57f0537d9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.151400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba803738-7d57-453f-9d79-e1314d1bfc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.152197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba803738-7d57-453f-9d79-e1314d1bfc59 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.152612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba803738-7d57-453f-9d79-e1314d1bfc59 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.152893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba803738-7d57-453f-9d79-e1314d1bfc59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.163398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba803738-7d57-453f-9d79-e1314d1bfc59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ba803738-7d57-453f-9d79-e1314d1bfc59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:26.170914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:26.171843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:26.172803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 547/1092] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:26.178358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba803738-7d57-453f-9d79-e1314d1bfc59 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.231798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:26.239533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d8ddc-3be1-4d8c-89bd-35a57f0537d9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.248070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:26.272032 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33bcfd-ffb9-40ea-a057-6d2acf51f421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.272802 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33bcfd-ffb9-40ea-a057-6d2acf51f421 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.273082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33bcfd-ffb9-40ea-a057-6d2acf51f421 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.273358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33bcfd-ffb9-40ea-a057-6d2acf51f421 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.274079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33bcfd-ffb9-40ea-a057-6d2acf51f421 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:26.274382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33bcfd-ffb9-40ea-a057-6d2acf51f421 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:26.275357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba803738-7d57-453f-9d79-e1314d1bfc59 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.275721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33bcfd-ffb9-40ea-a057-6d2acf51f421 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.288553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 546/1093] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:26.293755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33bcfd-ffb9-40ea-a057-6d2acf51f421 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.311495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c3654-58e7-4ba2-863a-d2ee292f01c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.312166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c3654-58e7-4ba2-863a-d2ee292f01c4 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.312478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c3654-58e7-4ba2-863a-d2ee292f01c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.312753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c3654-58e7-4ba2-863a-d2ee292f01c4 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.313493 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c3654-58e7-4ba2-863a-d2ee292f01c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:26.313827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c3654-58e7-4ba2-863a-d2ee292f01c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:26.315183 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c3654-58e7-4ba2-863a-d2ee292f01c4 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.323129 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 548/1094] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:26.328736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c3654-58e7-4ba2-863a-d2ee292f01c4 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.394323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33bcfd-ffb9-40ea-a057-6d2acf51f421 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.426113 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c3654-58e7-4ba2-863a-d2ee292f01c4 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.429083 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66334fe-8331-4d15-83c8-5146e75a837e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.429868 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66334fe-8331-4d15-83c8-5146e75a837e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.430154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66334fe-8331-4d15-83c8-5146e75a837e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.430429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66334fe-8331-4d15-83c8-5146e75a837e None admin] PATH_INFO: `/v3/users/nova` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.437384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66334fe-8331-4d15-83c8-5146e75a837e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:26.438832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66334fe-8331-4d15-83c8-5146e75a837e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.441560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-b66334fe-8331-4d15-83c8-5146e75a837e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:38:26.452809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:26.455283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:26.456267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 547/1095] 10.222.0.23 () {56 vars in 1093 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:26.459639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9684a-d3f0-4cb5-bd80-bebc5dbd9258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.459777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66334fe-8331-4d15-83c8-5146e75a837e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.460531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9684a-d3f0-4cb5-bd80-bebc5dbd9258 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.460810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9684a-d3f0-4cb5-bd80-bebc5dbd9258 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.461079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9684a-d3f0-4cb5-bd80-bebc5dbd9258 None admin] PATH_INFO: `/v3/users/5c96ffe1dd444559936512197dd76902` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.462267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9684a-d3f0-4cb5-bd80-bebc5dbd9258 None admin] RBAC: Authorizing `identity:get_user(user_id=5c96ffe1dd444559936512197dd76902)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:26.463683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9684a-d3f0-4cb5-bd80-bebc5dbd9258 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.469752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 549/1096] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/users/5c96ffe1dd444559936512197dd76902 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:26.471252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9684a-d3f0-4cb5-bd80-bebc5dbd9258 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.488312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:26.560550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66334fe-8331-4d15-83c8-5146e75a837e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.569473 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9684a-d3f0-4cb5-bd80-bebc5dbd9258 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.593672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf460946-40de-4ecf-9a48-b755912e0dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.594265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf460946-40de-4ecf-9a48-b755912e0dd5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.594547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf460946-40de-4ecf-9a48-b755912e0dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.594836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf460946-40de-4ecf-9a48-b755912e0dd5 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.596197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf460946-40de-4ecf-9a48-b755912e0dd5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:26.596536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf460946-40de-4ecf-9a48-b755912e0dd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:26.597941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf460946-40de-4ecf-9a48-b755912e0dd5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.603209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e93b02-7c6d-4b80-a8f8-b7366d9d0931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.603855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e93b02-7c6d-4b80-a8f8-b7366d9d0931 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.604143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e93b02-7c6d-4b80-a8f8-b7366d9d0931 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.604452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e93b02-7c6d-4b80-a8f8-b7366d9d0931 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.615601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 548/1097] 10.222.0.23 () {56 vars in 1133 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:26.621184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf460946-40de-4ecf-9a48-b755912e0dd5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.657406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:26.672674 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e93b02-7c6d-4b80-a8f8-b7366d9d0931 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:26.674119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e93b02-7c6d-4b80-a8f8-b7366d9d0931 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.703482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:26.718132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf460946-40de-4ecf-9a48-b755912e0dd5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.751758 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cfbf4-f560-43d6-93f0-aae4856e450f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.752423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cfbf4-f560-43d6-93f0-aae4856e450f None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.752705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cfbf4-f560-43d6-93f0-aae4856e450f None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.752978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cfbf4-f560-43d6-93f0-aae4856e450f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.770820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 550/1098] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:38:26.776650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e93b02-7c6d-4b80-a8f8-b7366d9d0931 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.818766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cfbf4-f560-43d6-93f0-aae4856e450f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:26.820225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cfbf4-f560-43d6-93f0-aae4856e450f None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.858580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:26.873900 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e93b02-7c6d-4b80-a8f8-b7366d9d0931 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:26.904000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:26.908375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2298562-746a-4e4f-9789-b010efda6a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:26.909242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2298562-746a-4e4f-9789-b010efda6a11 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:26.909528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2298562-746a-4e4f-9789-b010efda6a11 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:26.909896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2298562-746a-4e4f-9789-b010efda6a11 None admin] PATH_INFO: `/v3/projects/71195db0f36c4a7da07582dad1ab5284` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:26.910970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2298562-746a-4e4f-9789-b010efda6a11 None admin] RBAC: Authorizing `identity:get_project(project_id=71195db0f36c4a7da07582dad1ab5284)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:26.912406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2298562-746a-4e4f-9789-b010efda6a11 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:26.914215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 551/1099] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/projects/71195db0f36c4a7da07582dad1ab5284 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:26.919588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 549/1100] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 15:38:26.920669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2298562-746a-4e4f-9789-b010efda6a11 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:26.925070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cfbf4-f560-43d6-93f0-aae4856e450f None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:27.020719 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2298562-746a-4e4f-9789-b010efda6a11 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:27.023283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cfbf4-f560-43d6-93f0-aae4856e450f None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:27.054270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2e1dd-3e65-4956-a7c5-6db6969b7989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:27.054909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2e1dd-3e65-4956-a7c5-6db6969b7989 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:27.055200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2e1dd-3e65-4956-a7c5-6db6969b7989 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:27.055477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2e1dd-3e65-4956-a7c5-6db6969b7989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:27.056304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2e1dd-3e65-4956-a7c5-6db6969b7989 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=71195db0f36c4a7da07582dad1ab5284, user.id=5c96ffe1dd444559936512197dd76902) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:27.056652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2e1dd-3e65-4956-a7c5-6db6969b7989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:27.057088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68ba04-9e9a-4d10-8647-4bfb40e0a341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:27.057909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68ba04-9e9a-4d10-8647-4bfb40e0a341 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:27.058085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2e1dd-3e65-4956-a7c5-6db6969b7989 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:27.058185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68ba04-9e9a-4d10-8647-4bfb40e0a341 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:27.058515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68ba04-9e9a-4d10-8647-4bfb40e0a341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:27.064440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 552/1101] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/role_assignments?user.id=5c96ffe1dd444559936512197dd76902&scope.project.id=71195db0f36c4a7da07582dad1ab5284&role.id=04c11dea81274a69974ad5c639f2c60b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:27.069608 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68ba04-9e9a-4d10-8647-4bfb40e0a341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:27.071172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68ba04-9e9a-4d10-8647-4bfb40e0a341 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-ac68ba04-9e9a-4d10-8647-4bfb40e0a341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:27.091358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:27.092449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:27.093987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:38:27.093987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:27.093987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:38:27.093987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:27.093987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 550/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:27.098725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2e1dd-3e65-4956-a7c5-6db6969b7989 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:27.118427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:27.205980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2e1dd-3e65-4956-a7c5-6db6969b7989 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:27.240202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8012e7-bc7a-4059-b76c-216562c60caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:27.240897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8012e7-bc7a-4059-b76c-216562c60caa None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:27.241207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8012e7-bc7a-4059-b76c-216562c60caa None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:27.241484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8012e7-bc7a-4059-b76c-216562c60caa None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:27.242213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8012e7-bc7a-4059-b76c-216562c60caa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:27.242515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8012e7-bc7a-4059-b76c-216562c60caa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:27.243882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8012e7-bc7a-4059-b76c-216562c60caa None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:27.256078 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 553/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:27.261447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68ba04-9e9a-4d10-8647-4bfb40e0a341 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:27.303869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:27.362999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68ba04-9e9a-4d10-8647-4bfb40e0a341 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:27.396417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d3d5c-54f5-4ac1-9eed-5d049a9a3d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:27.397141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d3d5c-54f5-4ac1-9eed-5d049a9a3d4d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:27.397419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d3d5c-54f5-4ac1-9eed-5d049a9a3d4d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:27.397690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d3d5c-54f5-4ac1-9eed-5d049a9a3d4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:27.404138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d3d5c-54f5-4ac1-9eed-5d049a9a3d4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:27.405591 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d3d5c-54f5-4ac1-9eed-5d049a9a3d4d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-9c0d3d5c-54f5-4ac1-9eed-5d049a9a3d4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:38:27.418769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:27.419830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:27.425972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:38:27.425972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:38:27.425972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:38:27.425972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:38:27.425972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:38:27.425972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:38:27.425972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:38:27.425972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:27.425972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 551/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:38:27.426487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8012e7-bc7a-4059-b76c-216562c60caa None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:27.501227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:27.523954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8012e7-bc7a-4059-b76c-216562c60caa None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:27.556850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd50b1-3251-4f0e-ab40-f96e91c26ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:27.557512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd50b1-3251-4f0e-ab40-f96e91c26ffa None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:27.557797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd50b1-3251-4f0e-ab40-f96e91c26ffa None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:27.558074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd50b1-3251-4f0e-ab40-f96e91c26ffa None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:27.558865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd50b1-3251-4f0e-ab40-f96e91c26ffa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:27.559178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd50b1-3251-4f0e-ab40-f96e91c26ffa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:27.560641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd50b1-3251-4f0e-ab40-f96e91c26ffa None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:27.572902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 554/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:38:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:27.579156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d3d5c-54f5-4ac1-9eed-5d049a9a3d4d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:27.674897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:27.678564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d3d5c-54f5-4ac1-9eed-5d049a9a3d4d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:27.710326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b7324-469a-400f-9b83-e36d6c2af867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:27.710975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7324-469a-400f-9b83-e36d6c2af867 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:27.711254 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7324-469a-400f-9b83-e36d6c2af867 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:27.711528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b7324-469a-400f-9b83-e36d6c2af867 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:27.712350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b7324-469a-400f-9b83-e36d6c2af867 None admin] RBAC: Adding query filter params (role.id=8ad5ca9a5ad04e7fbb93d1330cf468e2, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=b9129f0b362a4945a0eca7cc33571abd) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:27.712686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b7324-469a-400f-9b83-e36d6c2af867 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:27.714091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b7324-469a-400f-9b83-e36d6c2af867 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:27.719850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 552/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:38:27 2022] GET /identity/v3/role_assignments?user.id=b9129f0b362a4945a0eca7cc33571abd&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=8ad5ca9a5ad04e7fbb93d1330cf468e2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:29.294254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93e53-94bd-48fa-9575-ea3edfcb6dc7 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:29.294697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93e53-94bd-48fa-9575-ea3edfcb6dc7 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:29.294810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93e53-94bd-48fa-9575-ea3edfcb6dc7 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:29.296177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 555/1107] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:29.301459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dd417-47f6-418a-a4e4-e13ab38b0b4f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:29.301850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dd417-47f6-418a-a4e4-e13ab38b0b4f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:29.302023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dd417-47f6-418a-a4e4-e13ab38b0b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:29.318763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c52dd417-47f6-418a-a4e4-e13ab38b0b4f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:29.319939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c52dd417-47f6-418a-a4e4-e13ab38b0b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:29.383553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 553/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:29.408124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd93e53-94bd-48fa-9575-ea3edfcb6dc7 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:29.505392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd93e53-94bd-48fa-9575-ea3edfcb6dc7 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:29.538724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b904b-f004-4f45-83cd-eca31b63560d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:29.539443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b904b-f004-4f45-83cd-eca31b63560d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:29.539725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b904b-f004-4f45-83cd-eca31b63560d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:29.540002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b904b-f004-4f45-83cd-eca31b63560d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:29.541194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b904b-f004-4f45-83cd-eca31b63560d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:29.542622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b904b-f004-4f45-83cd-eca31b63560d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:29.544180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 556/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:29.550120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dd417-47f6-418a-a4e4-e13ab38b0b4f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:29.636491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:29.644969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dd417-47f6-418a-a4e4-e13ab38b0b4f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:29.676788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb1d9c-0c59-4ebb-b126-8336ed4405f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:29.677449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb1d9c-0c59-4ebb-b126-8336ed4405f4 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:29.677735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb1d9c-0c59-4ebb-b126-8336ed4405f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:29.678012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb1d9c-0c59-4ebb-b126-8336ed4405f4 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:29.678720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb1d9c-0c59-4ebb-b126-8336ed4405f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:29.680514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb1d9c-0c59-4ebb-b126-8336ed4405f4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:29.776849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 554/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:38:29 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:38:31.952277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf904945-1be4-4916-ad75-edb470ae9d01 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:31.952863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf904945-1be4-4916-ad75-edb470ae9d01 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:31.952863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf904945-1be4-4916-ad75-edb470ae9d01 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:31.954236 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 557/1111] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:31.959949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1198-5144-47ee-97c0-83f30dbce9b3 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:31.960390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1198-5144-47ee-97c0-83f30dbce9b3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:31.960592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1198-5144-47ee-97c0-83f30dbce9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:31.977879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-338a1198-5144-47ee-97c0-83f30dbce9b3 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:31.979055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-338a1198-5144-47ee-97c0-83f30dbce9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:32.043543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 555/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:32.066950 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf904945-1be4-4916-ad75-edb470ae9d01 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:32.070751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:32.165248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf904945-1be4-4916-ad75-edb470ae9d01 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:32.198254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8169f6-0247-4300-9f83-872b59e16ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:32.199092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8169f6-0247-4300-9f83-872b59e16ef9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:32.199380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8169f6-0247-4300-9f83-872b59e16ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:32.199661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8169f6-0247-4300-9f83-872b59e16ef9 None admin] PATH_INFO: `/v3/roles/f7e0a506c7894b20b2b65fedfd2244e5` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:32.201138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8169f6-0247-4300-9f83-872b59e16ef9 None admin] RBAC: Authorizing `identity:get_role(role_id=f7e0a506c7894b20b2b65fedfd2244e5)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:32.202552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8169f6-0247-4300-9f83-872b59e16ef9 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:32.203850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 558/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:32 2022] GET /identity/v3/roles/f7e0a506c7894b20b2b65fedfd2244e5 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:32.209620 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a1198-5144-47ee-97c0-83f30dbce9b3 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:32.285760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:32.306178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a1198-5144-47ee-97c0-83f30dbce9b3 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:32.339212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84dd78-40f8-41e9-bc23-7fcc2fd738ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:32.340075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84dd78-40f8-41e9-bc23-7fcc2fd738ce None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:32.340350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84dd78-40f8-41e9-bc23-7fcc2fd738ce None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:32.340713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84dd78-40f8-41e9-bc23-7fcc2fd738ce None admin] PATH_INFO: `/v3/users/3eb339b5586f44e3bc4ea2888831039d` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:32.354055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84dd78-40f8-41e9-bc23-7fcc2fd738ce None admin] RBAC: Authorizing `identity:get_user(user_id=3eb339b5586f44e3bc4ea2888831039d)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:32.355473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84dd78-40f8-41e9-bc23-7fcc2fd738ce None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:32.357127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 556/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:32 2022] GET /identity/v3/users/3eb339b5586f44e3bc4ea2888831039d => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:32.362848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8169f6-0247-4300-9f83-872b59e16ef9 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:32.429781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:32.457414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8169f6-0247-4300-9f83-872b59e16ef9 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:32.489476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8dda6-1098-4ebc-8c5a-ea9d78479998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:32.490106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8dda6-1098-4ebc-8c5a-ea9d78479998 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:32.490386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8dda6-1098-4ebc-8c5a-ea9d78479998 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:32.490659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8dda6-1098-4ebc-8c5a-ea9d78479998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:32.554184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8dda6-1098-4ebc-8c5a-ea9d78479998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:32.555610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8dda6-1098-4ebc-8c5a-ea9d78479998 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:32.619380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:32.648661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 559/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:38:32.654499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84dd78-40f8-41e9-bc23-7fcc2fd738ce None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:32.749847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84dd78-40f8-41e9-bc23-7fcc2fd738ce None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:32.783143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155ee02-8bb5-42e5-99c8-272bd4e5e0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:32.783992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155ee02-8bb5-42e5-99c8-272bd4e5e0e3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:32.784277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155ee02-8bb5-42e5-99c8-272bd4e5e0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:32.784583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155ee02-8bb5-42e5-99c8-272bd4e5e0e3 None admin] PATH_INFO: `/v3/projects/71195db0f36c4a7da07582dad1ab5284` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:32.785756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155ee02-8bb5-42e5-99c8-272bd4e5e0e3 None admin] RBAC: Authorizing `identity:get_project(project_id=71195db0f36c4a7da07582dad1ab5284)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:32.787165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155ee02-8bb5-42e5-99c8-272bd4e5e0e3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:32.788889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 557/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:38:32 2022] GET /identity/v3/projects/71195db0f36c4a7da07582dad1ab5284 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:32.794954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8dda6-1098-4ebc-8c5a-ea9d78479998 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:32.889241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8dda6-1098-4ebc-8c5a-ea9d78479998 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:32.922300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d2914-2858-49af-bc36-47b7175f8613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:32.922941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d2914-2858-49af-bc36-47b7175f8613 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:32.923730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d2914-2858-49af-bc36-47b7175f8613 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:32.924033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d2914-2858-49af-bc36-47b7175f8613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:32.924888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d2914-2858-49af-bc36-47b7175f8613 None admin] RBAC: Adding query filter params (role.id=f7e0a506c7894b20b2b65fedfd2244e5, scope.project.id=71195db0f36c4a7da07582dad1ab5284, user.id=3eb339b5586f44e3bc4ea2888831039d) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:32.925202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d2914-2858-49af-bc36-47b7175f8613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:32.926616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d2914-2858-49af-bc36-47b7175f8613 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:32.931919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 560/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:38:32 2022] GET /identity/v3/role_assignments?user.id=3eb339b5586f44e3bc4ea2888831039d&scope.project.id=71195db0f36c4a7da07582dad1ab5284&role.id=f7e0a506c7894b20b2b65fedfd2244e5 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:35.066970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ec64e-4d82-46e8-af47-eebd4f12b70b None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:35.067528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ec64e-4d82-46e8-af47-eebd4f12b70b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:35.067528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ec64e-4d82-46e8-af47-eebd4f12b70b None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:35.069171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 558/1118] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:35.074528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908e0d2-c7d2-4d80-8894-a04743f10312 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:35.074960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908e0d2-c7d2-4d80-8894-a04743f10312 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:35.075091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908e0d2-c7d2-4d80-8894-a04743f10312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:35.091243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0908e0d2-c7d2-4d80-8894-a04743f10312 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:35.092453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0908e0d2-c7d2-4d80-8894-a04743f10312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:35.120808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:35.157757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 561/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:35.181141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ec64e-4d82-46e8-af47-eebd4f12b70b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:35.184896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:35.280164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ec64e-4d82-46e8-af47-eebd4f12b70b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:35.313814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16de5ae-5553-4457-be98-94af9a1661e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:35.315220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de5ae-5553-4457-be98-94af9a1661e9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:35.315502 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de5ae-5553-4457-be98-94af9a1661e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:35.315776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de5ae-5553-4457-be98-94af9a1661e9 None admin] PATH_INFO: `/v3/roles/f7e0a506c7894b20b2b65fedfd2244e5` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:35.317321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16de5ae-5553-4457-be98-94af9a1661e9 None admin] RBAC: Authorizing `identity:get_role(role_id=f7e0a506c7894b20b2b65fedfd2244e5)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:35.318655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16de5ae-5553-4457-be98-94af9a1661e9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:35.320004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 559/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:35 2022] GET /identity/v3/roles/f7e0a506c7894b20b2b65fedfd2244e5 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:35.325894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908e0d2-c7d2-4d80-8894-a04743f10312 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:35.420606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908e0d2-c7d2-4d80-8894-a04743f10312 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:35.453424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad64ad-8637-442e-a133-4089a37e787e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:35.454282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad64ad-8637-442e-a133-4089a37e787e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:35.454566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad64ad-8637-442e-a133-4089a37e787e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:35.454870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad64ad-8637-442e-a133-4089a37e787e None admin] PATH_INFO: `/v3/users/3eb339b5586f44e3bc4ea2888831039d` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:35.456044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad64ad-8637-442e-a133-4089a37e787e None admin] RBAC: Authorizing `identity:get_user(user_id=3eb339b5586f44e3bc4ea2888831039d)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:35.457512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad64ad-8637-442e-a133-4089a37e787e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:35.459100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 562/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:35 2022] GET /identity/v3/users/3eb339b5586f44e3bc4ea2888831039d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:35.464757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16de5ae-5553-4457-be98-94af9a1661e9 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:35.531840 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:35.560504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16de5ae-5553-4457-be98-94af9a1661e9 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:35.593056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dbce9-7ad3-41e8-afc9-10ced8e88588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:35.593836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dbce9-7ad3-41e8-afc9-10ced8e88588 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:35.594126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dbce9-7ad3-41e8-afc9-10ced8e88588 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:35.594405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dbce9-7ad3-41e8-afc9-10ced8e88588 None admin] PATH_INFO: `/v3/projects/71195db0f36c4a7da07582dad1ab5284` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:35.596065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dbce9-7ad3-41e8-afc9-10ced8e88588 None admin] RBAC: Authorizing `identity:get_project(project_id=71195db0f36c4a7da07582dad1ab5284)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:35.597495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dbce9-7ad3-41e8-afc9-10ced8e88588 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:35.599149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 560/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:38:35 2022] GET /identity/v3/projects/71195db0f36c4a7da07582dad1ab5284 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:35.604905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad64ad-8637-442e-a133-4089a37e787e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:35.637970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:35.700284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad64ad-8637-442e-a133-4089a37e787e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:35.732443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea98cb-ad85-4dbf-954f-c8eede9ef0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:35.733307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea98cb-ad85-4dbf-954f-c8eede9ef0af None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:35.733586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea98cb-ad85-4dbf-954f-c8eede9ef0af None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:35.733856 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea98cb-ad85-4dbf-954f-c8eede9ef0af None admin] PATH_INFO: `/v3/projects/71195db0f36c4a7da07582dad1ab5284/users/3eb339b5586f44e3bc4ea2888831039d/roles/f7e0a506c7894b20b2b65fedfd2244e5` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:35.736567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea98cb-ad85-4dbf-954f-c8eede9ef0af None admin] RBAC: Authorizing `identity:create_grant(project_id=71195db0f36c4a7da07582dad1ab5284, user_id=3eb339b5586f44e3bc4ea2888831039d, role_id=f7e0a506c7894b20b2b65fedfd2244e5)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:35.737979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea98cb-ad85-4dbf-954f-c8eede9ef0af None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:35.821794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 563/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:38:35 2022] PUT /identity/v3/projects/71195db0f36c4a7da07582dad1ab5284/users/3eb339b5586f44e3bc4ea2888831039d/roles/f7e0a506c7894b20b2b65fedfd2244e5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:38:37.961080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8552d8-8920-41a5-95fb-9a3504553c1e None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:37.961606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8552d8-8920-41a5-95fb-9a3504553c1e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:37.961606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8552d8-8920-41a5-95fb-9a3504553c1e None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:37.962984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 561/1124] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:37.968085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb3db6-531c-4d8e-82d5-b533c3501628 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:37.968545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb3db6-531c-4d8e-82d5-b533c3501628 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:37.968673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb3db6-531c-4d8e-82d5-b533c3501628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:37.984914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-7dbb3db6-531c-4d8e-82d5-b533c3501628 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:37.986100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7dbb3db6-531c-4d8e-82d5-b533c3501628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:38.051137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 564/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:38.074540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8552d8-8920-41a5-95fb-9a3504553c1e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:38.172542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8552d8-8920-41a5-95fb-9a3504553c1e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:38.205500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc5880-0ae7-4953-888a-d790f934220e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:38.206337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc5880-0ae7-4953-888a-d790f934220e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:38.206623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc5880-0ae7-4953-888a-d790f934220e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:38.206900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc5880-0ae7-4953-888a-d790f934220e None admin] PATH_INFO: `/v3/roles/f7e0a506c7894b20b2b65fedfd2244e5` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:38.208384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc5880-0ae7-4953-888a-d790f934220e None admin] RBAC: Authorizing `identity:get_role(role_id=f7e0a506c7894b20b2b65fedfd2244e5)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:38.209747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc5880-0ae7-4953-888a-d790f934220e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:38.211137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 562/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:38 2022] GET /identity/v3/roles/f7e0a506c7894b20b2b65fedfd2244e5 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:38.216661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb3db6-531c-4d8e-82d5-b533c3501628 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:38.240556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:38.314666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb3db6-531c-4d8e-82d5-b533c3501628 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:38.347485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f829703-dc8e-4626-998f-f6ad6a7d2f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:38.348347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f829703-dc8e-4626-998f-f6ad6a7d2f8d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:38.348677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f829703-dc8e-4626-998f-f6ad6a7d2f8d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:38.348953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f829703-dc8e-4626-998f-f6ad6a7d2f8d None admin] PATH_INFO: `/v3/users/3eb339b5586f44e3bc4ea2888831039d` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:38.350125 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f829703-dc8e-4626-998f-f6ad6a7d2f8d None admin] RBAC: Authorizing `identity:get_user(user_id=3eb339b5586f44e3bc4ea2888831039d)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:38.351575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f829703-dc8e-4626-998f-f6ad6a7d2f8d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:38.353229 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 565/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:38 2022] GET /identity/v3/users/3eb339b5586f44e3bc4ea2888831039d => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:38.358958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc5880-0ae7-4953-888a-d790f934220e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:38.370005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:38.454942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc5880-0ae7-4953-888a-d790f934220e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:38.487259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6a2c8-39e7-4ae4-890f-cddd65e28f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:38.487835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6a2c8-39e7-4ae4-890f-cddd65e28f01 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:38.488114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6a2c8-39e7-4ae4-890f-cddd65e28f01 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:38.488472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6a2c8-39e7-4ae4-890f-cddd65e28f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:38.552600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6a2c8-39e7-4ae4-890f-cddd65e28f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:38.554020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6a2c8-39e7-4ae4-890f-cddd65e28f01 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:38.562957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:38.647607 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 563/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:38:38.653396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f829703-dc8e-4626-998f-f6ad6a7d2f8d None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:38.740657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:38.748922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f829703-dc8e-4626-998f-f6ad6a7d2f8d None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:38.780565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4658115-ff23-49b7-b132-97b3942ae5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:38.781446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4658115-ff23-49b7-b132-97b3942ae5a9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:38.781728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4658115-ff23-49b7-b132-97b3942ae5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:38.782001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4658115-ff23-49b7-b132-97b3942ae5a9 None admin] PATH_INFO: `/v3/projects/71195db0f36c4a7da07582dad1ab5284` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:38.783694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4658115-ff23-49b7-b132-97b3942ae5a9 None admin] RBAC: Authorizing `identity:get_project(project_id=71195db0f36c4a7da07582dad1ab5284)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:38.785128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4658115-ff23-49b7-b132-97b3942ae5a9 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:38.787216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 566/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:38:38 2022] GET /identity/v3/projects/71195db0f36c4a7da07582dad1ab5284 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:38.792771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6a2c8-39e7-4ae4-890f-cddd65e28f01 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:38.886988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6a2c8-39e7-4ae4-890f-cddd65e28f01 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:38.897119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:38.920518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211ffc5-e2c8-4032-b657-f54263cb37d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:38.921153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211ffc5-e2c8-4032-b657-f54263cb37d3 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:38.921432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211ffc5-e2c8-4032-b657-f54263cb37d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:38.921703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211ffc5-e2c8-4032-b657-f54263cb37d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:38.922515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211ffc5-e2c8-4032-b657-f54263cb37d3 None admin] RBAC: Adding query filter params (role.id=f7e0a506c7894b20b2b65fedfd2244e5, scope.project.id=71195db0f36c4a7da07582dad1ab5284, user.id=3eb339b5586f44e3bc4ea2888831039d) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:38.922822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211ffc5-e2c8-4032-b657-f54263cb37d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:38.924300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211ffc5-e2c8-4032-b657-f54263cb37d3 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:38.930439 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 564/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:38:38 2022] GET /identity/v3/role_assignments?user.id=3eb339b5586f44e3bc4ea2888831039d&scope.project.id=71195db0f36c4a7da07582dad1ab5284&role.id=f7e0a506c7894b20b2b65fedfd2244e5 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:41.114084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0086f-2eb9-41f2-8034-6baa1a7ce5e9 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:41.114601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0086f-2eb9-41f2-8034-6baa1a7ce5e9 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:41.114691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0086f-2eb9-41f2-8034-6baa1a7ce5e9 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:41.116012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 567/1131] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:41.121143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54384d98-0d58-4e18-b20a-1c52f89bc2a5 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:41.121553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54384d98-0d58-4e18-b20a-1c52f89bc2a5 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:41.121694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54384d98-0d58-4e18-b20a-1c52f89bc2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:41.138105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-54384d98-0d58-4e18-b20a-1c52f89bc2a5 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:41.139529 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-54384d98-0d58-4e18-b20a-1c52f89bc2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:41.204640 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 565/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:41.228560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0086f-2eb9-41f2-8034-6baa1a7ce5e9 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:41.326938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0086f-2eb9-41f2-8034-6baa1a7ce5e9 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:41.359446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0c68e-1f31-4257-8081-d77c7e22209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:41.360222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0c68e-1f31-4257-8081-d77c7e22209d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:41.360542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0c68e-1f31-4257-8081-d77c7e22209d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:41.360821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0c68e-1f31-4257-8081-d77c7e22209d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:41.361999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0c68e-1f31-4257-8081-d77c7e22209d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:41.363427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0c68e-1f31-4257-8081-d77c7e22209d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:41.365076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 568/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:41.370847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54384d98-0d58-4e18-b20a-1c52f89bc2a5 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:41.465539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54384d98-0d58-4e18-b20a-1c52f89bc2a5 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:41.466987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:41.498505 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f7567-faa8-4fcb-b916-905bf4e1956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:41.499151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7567-faa8-4fcb-b916-905bf4e1956c None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:41.499433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7567-faa8-4fcb-b916-905bf4e1956c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:41.499705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7567-faa8-4fcb-b916-905bf4e1956c None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:41.500423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f7567-faa8-4fcb-b916-905bf4e1956c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:41.502350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f7567-faa8-4fcb-b916-905bf4e1956c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:41.583874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 566/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:38:41 2022] POST /identity/v3/projects => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:38:43.750140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e583d3-7aaa-468d-9965-52e7b06dcb67 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:43.750678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e583d3-7aaa-468d-9965-52e7b06dcb67 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:43.750787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e583d3-7aaa-468d-9965-52e7b06dcb67 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:43.752670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 569/1135] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:43.757788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8524-5905-41b6-8fe9-e2392dfc08a3 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:43.758168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8524-5905-41b6-8fe9-e2392dfc08a3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:43.758357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8524-5905-41b6-8fe9-e2392dfc08a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:43.775560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b62c8524-5905-41b6-8fe9-e2392dfc08a3 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:43.776768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b62c8524-5905-41b6-8fe9-e2392dfc08a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:43.840899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 567/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:43.865177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e583d3-7aaa-468d-9965-52e7b06dcb67 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:43.893568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:43.964016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e583d3-7aaa-468d-9965-52e7b06dcb67 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:43.995799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879c7d1-cc2a-43fb-a721-3c7a57720f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:43.996597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879c7d1-cc2a-43fb-a721-3c7a57720f1b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:43.996880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879c7d1-cc2a-43fb-a721-3c7a57720f1b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:43.997154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879c7d1-cc2a-43fb-a721-3c7a57720f1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:43.998308 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879c7d1-cc2a-43fb-a721-3c7a57720f1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:43.999739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879c7d1-cc2a-43fb-a721-3c7a57720f1b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:44.001392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 570/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:38:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:44.007426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c8524-5905-41b6-8fe9-e2392dfc08a3 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:44.089217 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:44.103960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c8524-5905-41b6-8fe9-e2392dfc08a3 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:44.137175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7f26f-2850-4467-a263-9865eb5fdaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:44.137840 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7f26f-2850-4467-a263-9865eb5fdaed None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:44.138123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7f26f-2850-4467-a263-9865eb5fdaed None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:44.138398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7f26f-2850-4467-a263-9865eb5fdaed None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:44.139362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7f26f-2850-4467-a263-9865eb5fdaed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:44.140817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7f26f-2850-4467-a263-9865eb5fdaed None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:44.245460 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 568/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:38:44 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:38:46.461231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0c58b-6340-4ac1-a58a-8cfdd0ce4ecf None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:46.461906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0c58b-6340-4ac1-a58a-8cfdd0ce4ecf None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:46.461906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0c58b-6340-4ac1-a58a-8cfdd0ce4ecf None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:46.463143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 571/1139] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:46.468581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6a506-0812-418a-9ec4-1183a0f28a50 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:46.468977 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6a506-0812-418a-9ec4-1183a0f28a50 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:46.469138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6a506-0812-418a-9ec4-1183a0f28a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:46.485278 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-61f6a506-0812-418a-9ec4-1183a0f28a50 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:46.486933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-61f6a506-0812-418a-9ec4-1183a0f28a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:46.550484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 569/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:46.573787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0c58b-6340-4ac1-a58a-8cfdd0ce4ecf None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:46.664012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:46.670423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0c58b-6340-4ac1-a58a-8cfdd0ce4ecf None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:46.703601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25329c7-fdb3-4d07-9e29-3116e9bd8ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:46.704405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25329c7-fdb3-4d07-9e29-3116e9bd8ed7 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:46.704693 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25329c7-fdb3-4d07-9e29-3116e9bd8ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:46.704973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25329c7-fdb3-4d07-9e29-3116e9bd8ed7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:46.714661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25329c7-fdb3-4d07-9e29-3116e9bd8ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-c25329c7-fdb3-4d07-9e29-3116e9bd8ed7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:46.722140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:46.723216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:46.724110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 572/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:38:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:46.729238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6a506-0812-418a-9ec4-1183a0f28a50 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:46.795091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:46.824520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6a506-0812-418a-9ec4-1183a0f28a50 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:46.858666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4daed-ef99-4378-bc54-2f219bf50f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:46.859326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4daed-ef99-4378-bc54-2f219bf50f6d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:46.859608 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4daed-ef99-4378-bc54-2f219bf50f6d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:46.859890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4daed-ef99-4378-bc54-2f219bf50f6d None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:46.860663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4daed-ef99-4378-bc54-2f219bf50f6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:46.860970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4daed-ef99-4378-bc54-2f219bf50f6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:46.862857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4daed-ef99-4378-bc54-2f219bf50f6d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:46.870556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 570/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:38:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:46.876293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25329c7-fdb3-4d07-9e29-3116e9bd8ed7 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:46.972298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25329c7-fdb3-4d07-9e29-3116e9bd8ed7 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:47.004130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8162dff-2c6c-44dd-8226-f1ef526efc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:47.005010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8162dff-2c6c-44dd-8226-f1ef526efc00 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:47.005298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8162dff-2c6c-44dd-8226-f1ef526efc00 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:47.005579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8162dff-2c6c-44dd-8226-f1ef526efc00 None admin] PATH_INFO: `/v3/users/5601606fa9ad4535a68dbcdacd380219` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:47.018500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8162dff-2c6c-44dd-8226-f1ef526efc00 None admin] RBAC: Authorizing `identity:get_user(user_id=5601606fa9ad4535a68dbcdacd380219)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:47.019926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8162dff-2c6c-44dd-8226-f1ef526efc00 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:47.021545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 573/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:46 2022] GET /identity/v3/users/5601606fa9ad4535a68dbcdacd380219 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:47.026995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4daed-ef99-4378-bc54-2f219bf50f6d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:47.125730 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4daed-ef99-4378-bc54-2f219bf50f6d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:47.149867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:47.154551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e89796-48c0-47b7-a9a5-d952355831bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:47.155216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e89796-48c0-47b7-a9a5-d952355831bd None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:47.155498 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e89796-48c0-47b7-a9a5-d952355831bd None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:47.155773 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e89796-48c0-47b7-a9a5-d952355831bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:47.222149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e89796-48c0-47b7-a9a5-d952355831bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:47.223643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e89796-48c0-47b7-a9a5-d952355831bd None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:47.317526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 571/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:38:47.323347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8162dff-2c6c-44dd-8226-f1ef526efc00 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:47.339029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:47.420266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8162dff-2c6c-44dd-8226-f1ef526efc00 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:47.451954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35d409-9011-4991-8524-5d22d580cbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:47.452826 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35d409-9011-4991-8524-5d22d580cbbc None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:47.453108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35d409-9011-4991-8524-5d22d580cbbc None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:47.453381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35d409-9011-4991-8524-5d22d580cbbc None admin] PATH_INFO: `/v3/projects/9c29be5de67b45a2ab0f1c2e438e3b56` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:47.454532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35d409-9011-4991-8524-5d22d580cbbc None admin] RBAC: Authorizing `identity:get_project(project_id=9c29be5de67b45a2ab0f1c2e438e3b56)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:47.456510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35d409-9011-4991-8524-5d22d580cbbc None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:47.458155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 574/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:38:47 2022] GET /identity/v3/projects/9c29be5de67b45a2ab0f1c2e438e3b56 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:47.464024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e89796-48c0-47b7-a9a5-d952355831bd None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:47.493497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:47.558320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e89796-48c0-47b7-a9a5-d952355831bd None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:47.591049 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865a59-81bd-4a8d-9738-9ab23320ffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:47.591690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865a59-81bd-4a8d-9738-9ab23320ffb0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:47.591972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865a59-81bd-4a8d-9738-9ab23320ffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:47.592248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865a59-81bd-4a8d-9738-9ab23320ffb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:47.593100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865a59-81bd-4a8d-9738-9ab23320ffb0 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=9c29be5de67b45a2ab0f1c2e438e3b56, user.id=5601606fa9ad4535a68dbcdacd380219) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:47.593412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865a59-81bd-4a8d-9738-9ab23320ffb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:47.594818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865a59-81bd-4a8d-9738-9ab23320ffb0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:47.600039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 572/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:38:47 2022] GET /identity/v3/role_assignments?user.id=5601606fa9ad4535a68dbcdacd380219&scope.project.id=9c29be5de67b45a2ab0f1c2e438e3b56&role.id=04c11dea81274a69974ad5c639f2c60b => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:49.733259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f1bc8-6f4f-45f0-a437-3d46fb390f88 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:49.733752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f1bc8-6f4f-45f0-a437-3d46fb390f88 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:49.733752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f1bc8-6f4f-45f0-a437-3d46fb390f88 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:49.735110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 575/1147] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:49.740539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640c62d-7c12-4295-813e-79757d85edde None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:49.740972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640c62d-7c12-4295-813e-79757d85edde None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:49.741107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640c62d-7c12-4295-813e-79757d85edde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:49.758011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2640c62d-7c12-4295-813e-79757d85edde None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:49.759199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2640c62d-7c12-4295-813e-79757d85edde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:49.823054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 573/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:49.845817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f1bc8-6f4f-45f0-a437-3d46fb390f88 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:49.925716 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:49.944204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f1bc8-6f4f-45f0-a437-3d46fb390f88 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:49.977051 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3794d88-401a-4151-9723-e2cb354dbfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:49.977833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3794d88-401a-4151-9723-e2cb354dbfa5 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:49.978117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3794d88-401a-4151-9723-e2cb354dbfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:49.978391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3794d88-401a-4151-9723-e2cb354dbfa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:49.989970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3794d88-401a-4151-9723-e2cb354dbfa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-b3794d88-401a-4151-9723-e2cb354dbfa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:49.996322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:49.997454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:49.998532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 576/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:38:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:50.004000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640c62d-7c12-4295-813e-79757d85edde None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:50.029806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:50.099062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640c62d-7c12-4295-813e-79757d85edde None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:50.132884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989fb39-e7ac-4b9f-819e-aea843d1d156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:50.133543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989fb39-e7ac-4b9f-819e-aea843d1d156 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:50.133825 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989fb39-e7ac-4b9f-819e-aea843d1d156 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:50.134103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989fb39-e7ac-4b9f-819e-aea843d1d156 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:50.134842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989fb39-e7ac-4b9f-819e-aea843d1d156 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:50.135150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989fb39-e7ac-4b9f-819e-aea843d1d156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:50.136536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989fb39-e7ac-4b9f-819e-aea843d1d156 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:50.144654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 574/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:38:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:50.150538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3794d88-401a-4151-9723-e2cb354dbfa5 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:50.248033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3794d88-401a-4151-9723-e2cb354dbfa5 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:50.280879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de4133-d677-455b-bc63-1c5ac586d5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:50.281750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de4133-d677-455b-bc63-1c5ac586d5f9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:50.282031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de4133-d677-455b-bc63-1c5ac586d5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:50.282302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de4133-d677-455b-bc63-1c5ac586d5f9 None admin] PATH_INFO: `/v3/users/5601606fa9ad4535a68dbcdacd380219` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:50.284106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de4133-d677-455b-bc63-1c5ac586d5f9 None admin] RBAC: Authorizing `identity:get_user(user_id=5601606fa9ad4535a68dbcdacd380219)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:50.285579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de4133-d677-455b-bc63-1c5ac586d5f9 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:50.287195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 577/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:50 2022] GET /identity/v3/users/5601606fa9ad4535a68dbcdacd380219 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:50.292751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989fb39-e7ac-4b9f-819e-aea843d1d156 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:50.378525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:50.390211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989fb39-e7ac-4b9f-819e-aea843d1d156 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:50.423584 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0525c8d-6a0d-4173-a487-6e1e91393dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:50.424470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0525c8d-6a0d-4173-a487-6e1e91393dda None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:50.424756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0525c8d-6a0d-4173-a487-6e1e91393dda None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:50.425032 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0525c8d-6a0d-4173-a487-6e1e91393dda None admin] PATH_INFO: `/v3/projects/9c29be5de67b45a2ab0f1c2e438e3b56` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:50.426196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0525c8d-6a0d-4173-a487-6e1e91393dda None admin] RBAC: Authorizing `identity:get_project(project_id=9c29be5de67b45a2ab0f1c2e438e3b56)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:50.427611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0525c8d-6a0d-4173-a487-6e1e91393dda None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:50.429310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 575/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:38:50 2022] GET /identity/v3/projects/9c29be5de67b45a2ab0f1c2e438e3b56 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:50.435117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de4133-d677-455b-bc63-1c5ac586d5f9 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:50.448038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:50.532694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de4133-d677-455b-bc63-1c5ac586d5f9 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:50.564468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cb4cb-286c-40fe-88ea-4477dcb5060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:50.565853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cb4cb-286c-40fe-88ea-4477dcb5060f None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:50.566139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cb4cb-286c-40fe-88ea-4477dcb5060f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:50.566415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cb4cb-286c-40fe-88ea-4477dcb5060f None admin] PATH_INFO: `/v3/projects/9c29be5de67b45a2ab0f1c2e438e3b56/users/5601606fa9ad4535a68dbcdacd380219/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:50.568433 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28cb4cb-286c-40fe-88ea-4477dcb5060f None admin] RBAC: Authorizing `identity:create_grant(project_id=9c29be5de67b45a2ab0f1c2e438e3b56, user_id=5601606fa9ad4535a68dbcdacd380219, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:50.569858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28cb4cb-286c-40fe-88ea-4477dcb5060f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:50.645971 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 578/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:38:50 2022] PUT /identity/v3/projects/9c29be5de67b45a2ab0f1c2e438e3b56/users/5601606fa9ad4535a68dbcdacd380219/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:38:52.779407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee1f14-8965-4fb0-bd2c-455a8be7f5c4 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:52.779916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee1f14-8965-4fb0-bd2c-455a8be7f5c4 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:52.780031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee1f14-8965-4fb0-bd2c-455a8be7f5c4 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:52.781368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 576/1154] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:52.786474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e908e2-c742-4d4c-9c6d-7f8bce6374b3 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:52.786878 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e908e2-c742-4d4c-9c6d-7f8bce6374b3 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:52.787030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e908e2-c742-4d4c-9c6d-7f8bce6374b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:52.803201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-35e908e2-c742-4d4c-9c6d-7f8bce6374b3 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:52.804377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-35e908e2-c742-4d4c-9c6d-7f8bce6374b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:52.864929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:52.869318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 579/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:52.892779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee1f14-8965-4fb0-bd2c-455a8be7f5c4 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:52.991059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee1f14-8965-4fb0-bd2c-455a8be7f5c4 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:53.023801 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774c700-1b31-49f2-8a3d-dcb3a846ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:53.024599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774c700-1b31-49f2-8a3d-dcb3a846ebff None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:53.024887 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774c700-1b31-49f2-8a3d-dcb3a846ebff None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:53.025163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774c700-1b31-49f2-8a3d-dcb3a846ebff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:53.035031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774c700-1b31-49f2-8a3d-dcb3a846ebff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-7774c700-1b31-49f2-8a3d-dcb3a846ebff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:38:53.042544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:53.043609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:38:53.044641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 577/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:38:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:53.049943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e908e2-c742-4d4c-9c6d-7f8bce6374b3 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:53.144377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e908e2-c742-4d4c-9c6d-7f8bce6374b3 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:53.177368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6cfdc-0724-4ed3-b51f-70106e7e8c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:53.178026 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6cfdc-0724-4ed3-b51f-70106e7e8c0f None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:53.178312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6cfdc-0724-4ed3-b51f-70106e7e8c0f None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:53.178594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6cfdc-0724-4ed3-b51f-70106e7e8c0f None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:53.179330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6cfdc-0724-4ed3-b51f-70106e7e8c0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:53.179641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6cfdc-0724-4ed3-b51f-70106e7e8c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:53.181038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6cfdc-0724-4ed3-b51f-70106e7e8c0f None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:53.188805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 580/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:38:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:53.194951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774c700-1b31-49f2-8a3d-dcb3a846ebff None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:53.213577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:53.296993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774c700-1b31-49f2-8a3d-dcb3a846ebff None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:53.329682 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5dd8e-21d5-4d72-a161-8e7ee3d616fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:53.330589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5dd8e-21d5-4d72-a161-8e7ee3d616fa None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:53.330874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5dd8e-21d5-4d72-a161-8e7ee3d616fa None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:53.331150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5dd8e-21d5-4d72-a161-8e7ee3d616fa None admin] PATH_INFO: `/v3/users/5601606fa9ad4535a68dbcdacd380219` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:53.332331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5dd8e-21d5-4d72-a161-8e7ee3d616fa None admin] RBAC: Authorizing `identity:get_user(user_id=5601606fa9ad4535a68dbcdacd380219)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:53.333783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5dd8e-21d5-4d72-a161-8e7ee3d616fa None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:53.335398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 578/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:38:53 2022] GET /identity/v3/users/5601606fa9ad4535a68dbcdacd380219 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:53.341238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6cfdc-0724-4ed3-b51f-70106e7e8c0f None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:53.400958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:53.439388 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6cfdc-0724-4ed3-b51f-70106e7e8c0f None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:53.471470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2348de-1801-4f92-ab13-34287e775128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:53.472106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2348de-1801-4f92-ab13-34287e775128 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:53.472413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2348de-1801-4f92-ab13-34287e775128 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:53.472696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2348de-1801-4f92-ab13-34287e775128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:53.536448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2348de-1801-4f92-ab13-34287e775128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:53.537879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2348de-1801-4f92-ab13-34287e775128 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:53.594256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:53.631820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 581/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:38:53.637458 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5dd8e-21d5-4d72-a161-8e7ee3d616fa None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:53.719308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:53.734602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5dd8e-21d5-4d72-a161-8e7ee3d616fa None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:53.767334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1160c1-d084-449b-a42d-ca185b60a111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:53.768728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1160c1-d084-449b-a42d-ca185b60a111 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:53.769004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1160c1-d084-449b-a42d-ca185b60a111 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:53.769272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1160c1-d084-449b-a42d-ca185b60a111 None admin] PATH_INFO: `/v3/projects/9c29be5de67b45a2ab0f1c2e438e3b56` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:53.770437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1160c1-d084-449b-a42d-ca185b60a111 None admin] RBAC: Authorizing `identity:get_project(project_id=9c29be5de67b45a2ab0f1c2e438e3b56)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:53.771831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1160c1-d084-449b-a42d-ca185b60a111 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:53.773506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 579/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:38:53 2022] GET /identity/v3/projects/9c29be5de67b45a2ab0f1c2e438e3b56 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:53.779526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2348de-1801-4f92-ab13-34287e775128 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:53.874087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2348de-1801-4f92-ab13-34287e775128 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:53.906436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42239645-4b1d-41fb-a657-de7d8ddba012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:53.907072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42239645-4b1d-41fb-a657-de7d8ddba012 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:53.907355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42239645-4b1d-41fb-a657-de7d8ddba012 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:53.907630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42239645-4b1d-41fb-a657-de7d8ddba012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:53.908522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42239645-4b1d-41fb-a657-de7d8ddba012 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=9c29be5de67b45a2ab0f1c2e438e3b56, user.id=5601606fa9ad4535a68dbcdacd380219) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:53.909347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42239645-4b1d-41fb-a657-de7d8ddba012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:53.910767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42239645-4b1d-41fb-a657-de7d8ddba012 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:53.916287 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 582/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:38:53 2022] GET /identity/v3/role_assignments?user.id=5601606fa9ad4535a68dbcdacd380219&scope.project.id=9c29be5de67b45a2ab0f1c2e438e3b56&role.id=04c11dea81274a69974ad5c639f2c60b => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:38:56.082312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b87c6-4a9e-491c-9989-826fe716dfb0 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:56.082832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b87c6-4a9e-491c-9989-826fe716dfb0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:56.082945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b87c6-4a9e-491c-9989-826fe716dfb0 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:56.084233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 580/1162] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:56.089338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5985848-8340-4d9a-8d0e-d087f2be5879 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:56.089746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5985848-8340-4d9a-8d0e-d087f2be5879 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:56.089899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5985848-8340-4d9a-8d0e-d087f2be5879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:56.106024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-f5985848-8340-4d9a-8d0e-d087f2be5879 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:56.107206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f5985848-8340-4d9a-8d0e-d087f2be5879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:56.126812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:56.171969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 583/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:56.194517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b87c6-4a9e-491c-9989-826fe716dfb0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:56.293400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b87c6-4a9e-491c-9989-826fe716dfb0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:56.327650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a9555-7844-48ad-9ac6-1275caa1fb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:56.328588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a9555-7844-48ad-9ac6-1275caa1fb07 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:56.329005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a9555-7844-48ad-9ac6-1275caa1fb07 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:56.329357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a9555-7844-48ad-9ac6-1275caa1fb07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:56.353722 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:56.398094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a9555-7844-48ad-9ac6-1275caa1fb07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:56.399526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a9555-7844-48ad-9ac6-1275caa1fb07 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:56.493455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 581/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:38:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:38:56.499396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5985848-8340-4d9a-8d0e-d087f2be5879 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:56.594236 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5985848-8340-4d9a-8d0e-d087f2be5879 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:56.626584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb860d-4d09-4617-ad92-f48a68f94f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:56.627310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb860d-4d09-4617-ad92-f48a68f94f10 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:56.628148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb860d-4d09-4617-ad92-f48a68f94f10 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:56.628449 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb860d-4d09-4617-ad92-f48a68f94f10 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:56.634816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb860d-4d09-4617-ad92-f48a68f94f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:56.636236 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb860d-4d09-4617-ad92-f48a68f94f10 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-37cb860d-4d09-4617-ad92-f48a68f94f10 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:38:56.650875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:38:56.651939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:38:56.652904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 584/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 23 15:38:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:38:56.657936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a9555-7844-48ad-9ac6-1275caa1fb07 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:56.708499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:56.753681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a9555-7844-48ad-9ac6-1275caa1fb07 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:56.786547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b3b82-fffa-4bc0-b13e-eed2644e5b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:56.787190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b3b82-fffa-4bc0-b13e-eed2644e5b69 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:56.787471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b3b82-fffa-4bc0-b13e-eed2644e5b69 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:56.787742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b3b82-fffa-4bc0-b13e-eed2644e5b69 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:56.788473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b3b82-fffa-4bc0-b13e-eed2644e5b69 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:38:56.788776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b3b82-fffa-4bc0-b13e-eed2644e5b69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:56.790193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b3b82-fffa-4bc0-b13e-eed2644e5b69 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:56.797007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 582/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Mon May 23 15:38:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:38:58.966884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527da44d-7126-4d03-a30d-cdc7ac7329b2 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:58.967421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527da44d-7126-4d03-a30d-cdc7ac7329b2 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:58.967421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527da44d-7126-4d03-a30d-cdc7ac7329b2 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:58.968846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 585/1167] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:38:58.974052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5e14-8046-4290-b331-5a333eaccffd None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:58.974437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5e14-8046-4290-b331-5a333eaccffd None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:58.974621 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5e14-8046-4290-b331-5a333eaccffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:58.991432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-809f5e14-8046-4290-b331-5a333eaccffd None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:38:58.992653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-809f5e14-8046-4290-b331-5a333eaccffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:38:59.057269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 583/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:38:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:38:59.080255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527da44d-7126-4d03-a30d-cdc7ac7329b2 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:38:59.098416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:38:59.178492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527da44d-7126-4d03-a30d-cdc7ac7329b2 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:38:59.210652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59da5ca-ea2a-42bd-9c27-c98dfce01b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:38:59.211294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59da5ca-ea2a-42bd-9c27-c98dfce01b06 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:38:59.211574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59da5ca-ea2a-42bd-9c27-c98dfce01b06 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:38:59.211849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59da5ca-ea2a-42bd-9c27-c98dfce01b06 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:38:59.212610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59da5ca-ea2a-42bd-9c27-c98dfce01b06 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:38:59.214041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59da5ca-ea2a-42bd-9c27-c98dfce01b06 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:38:59.288996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 586/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Mon May 23 15:38:59 2022] POST /identity/v3/domains => generated 267 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:39:01.465880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08d005-f1f8-48f4-a02e-8133962bf60f None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:01.466440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08d005-f1f8-48f4-a02e-8133962bf60f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:01.466440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08d005-f1f8-48f4-a02e-8133962bf60f None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:01.467807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 584/1170] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:01.473627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046474fd-35b5-482f-b7ad-7e664c49644a None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:01.474043 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046474fd-35b5-482f-b7ad-7e664c49644a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:01.474192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046474fd-35b5-482f-b7ad-7e664c49644a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:01.491085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-046474fd-35b5-482f-b7ad-7e664c49644a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:01.492269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-046474fd-35b5-482f-b7ad-7e664c49644a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:01.557460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 587/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:39:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:39:01.579046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08d005-f1f8-48f4-a02e-8133962bf60f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:01.625319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:01.677525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08d005-f1f8-48f4-a02e-8133962bf60f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:01.709476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5eadd-954a-4f60-8f58-4cc1005846ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:01.710325 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5eadd-954a-4f60-8f58-4cc1005846ba None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:01.710608 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5eadd-954a-4f60-8f58-4cc1005846ba None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:01.710881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5eadd-954a-4f60-8f58-4cc1005846ba None admin] PATH_INFO: `/v3/domains/e3364acf76024b90b53d4f46b907c74b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:01.720835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5eadd-954a-4f60-8f58-4cc1005846ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=e3364acf76024b90b53d4f46b907c74b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:01.722841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5eadd-954a-4f60-8f58-4cc1005846ba None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:01.724453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 585/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Mon May 23 15:39:01 2022] GET /identity/v3/domains/e3364acf76024b90b53d4f46b907c74b => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:01.730226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046474fd-35b5-482f-b7ad-7e664c49644a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:01.738819 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:01.827497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046474fd-35b5-482f-b7ad-7e664c49644a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:01.860163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f42733-7484-411b-8669-f0a74cac5e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:01.860840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f42733-7484-411b-8669-f0a74cac5e84 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:01.861127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f42733-7484-411b-8669-f0a74cac5e84 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:01.861402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f42733-7484-411b-8669-f0a74cac5e84 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:01.862127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f42733-7484-411b-8669-f0a74cac5e84 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:01.863577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f42733-7484-411b-8669-f0a74cac5e84 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:01.938890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 588/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:39:01 2022] POST /identity/v3/projects => generated 361 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:39:04.135641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07083eb-a423-484e-b398-e5f4f3fa5b44 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:04.136197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07083eb-a423-484e-b398-e5f4f3fa5b44 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:04.136197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07083eb-a423-484e-b398-e5f4f3fa5b44 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:04.137573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 586/1174] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:04.142996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b7e94-b1fe-4ccc-9882-f6f9bb9b2066 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:04.143377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b7e94-b1fe-4ccc-9882-f6f9bb9b2066 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:04.143569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b7e94-b1fe-4ccc-9882-f6f9bb9b2066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:04.159844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b13b7e94-b1fe-4ccc-9882-f6f9bb9b2066 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:04.161058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b13b7e94-b1fe-4ccc-9882-f6f9bb9b2066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:04.211943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:04.224755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 589/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:39:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:39:04.249294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07083eb-a423-484e-b398-e5f4f3fa5b44 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:04.347824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07083eb-a423-484e-b398-e5f4f3fa5b44 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:04.354434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:04.380936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898dc6d-93e8-47f6-ab05-a38c1f476889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:04.381785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898dc6d-93e8-47f6-ab05-a38c1f476889 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:04.382063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898dc6d-93e8-47f6-ab05-a38c1f476889 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:04.382335 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898dc6d-93e8-47f6-ab05-a38c1f476889 None admin] PATH_INFO: `/v3/domains/e3364acf76024b90b53d4f46b907c74b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:04.383701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898dc6d-93e8-47f6-ab05-a38c1f476889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e3364acf76024b90b53d4f46b907c74b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:04.385729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898dc6d-93e8-47f6-ab05-a38c1f476889 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:04.387294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 587/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Mon May 23 15:39:04 2022] GET /identity/v3/domains/e3364acf76024b90b53d4f46b907c74b => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:04.393310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b7e94-b1fe-4ccc-9882-f6f9bb9b2066 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:04.486989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b7e94-b1fe-4ccc-9882-f6f9bb9b2066 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:04.519313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d1463-eb9f-4b8a-9a65-6b5b8ca1073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:04.519967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d1463-eb9f-4b8a-9a65-6b5b8ca1073d None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:04.520251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d1463-eb9f-4b8a-9a65-6b5b8ca1073d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:04.520559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d1463-eb9f-4b8a-9a65-6b5b8ca1073d None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:04.521250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d1463-eb9f-4b8a-9a65-6b5b8ca1073d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:04.522663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d1463-eb9f-4b8a-9a65-6b5b8ca1073d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:04.605379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 590/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:39:04 2022] POST /identity/v3/users => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:39:06.795478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2df996-f594-4aa4-a439-3850eed7019c None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:06.796016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2df996-f594-4aa4-a439-3850eed7019c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:06.796016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2df996-f594-4aa4-a439-3850eed7019c None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:06.797437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 588/1178] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:06.802690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfda117-4bbb-4ecd-9ca3-32d1bfb834ba None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:06.803119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfda117-4bbb-4ecd-9ca3-32d1bfb834ba None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:06.803249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfda117-4bbb-4ecd-9ca3-32d1bfb834ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:06.819965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-dbfda117-4bbb-4ecd-9ca3-32d1bfb834ba None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:06.821159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-dbfda117-4bbb-4ecd-9ca3-32d1bfb834ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:06.859478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:06.884773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 591/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:39:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:39:06.908468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2df996-f594-4aa4-a439-3850eed7019c None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:07.007580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2df996-f594-4aa4-a439-3850eed7019c None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:07.040267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5cff1-9d4e-4bc3-982a-9015c3fb9382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:07.041069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5cff1-9d4e-4bc3-982a-9015c3fb9382 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:07.041352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5cff1-9d4e-4bc3-982a-9015c3fb9382 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:07.041626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5cff1-9d4e-4bc3-982a-9015c3fb9382 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:07.051448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5cff1-9d4e-4bc3-982a-9015c3fb9382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-01e5cff1-9d4e-4bc3-982a-9015c3fb9382 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:39:07.058863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:39:07.059965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:39:07.061029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 589/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:39:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:39:07.066584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfda117-4bbb-4ecd-9ca3-32d1bfb834ba None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:07.161267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfda117-4bbb-4ecd-9ca3-32d1bfb834ba None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:07.194151 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ac353-53a1-4f0e-80c6-9ed53c11005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:07.194808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ac353-53a1-4f0e-80c6-9ed53c11005c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:07.195090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ac353-53a1-4f0e-80c6-9ed53c11005c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:07.195363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ac353-53a1-4f0e-80c6-9ed53c11005c None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:07.196105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ac353-53a1-4f0e-80c6-9ed53c11005c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:39:07.196439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ac353-53a1-4f0e-80c6-9ed53c11005c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:07.197807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ac353-53a1-4f0e-80c6-9ed53c11005c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:07.205536 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 592/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:39:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:07.211154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5cff1-9d4e-4bc3-982a-9015c3fb9382 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:07.244562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:07.310008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5cff1-9d4e-4bc3-982a-9015c3fb9382 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:07.342462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7fb9-059c-4d6c-b2fa-4314ff09af88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:07.343314 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7fb9-059c-4d6c-b2fa-4314ff09af88 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:07.343595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7fb9-059c-4d6c-b2fa-4314ff09af88 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:07.343867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7fb9-059c-4d6c-b2fa-4314ff09af88 None admin] PATH_INFO: `/v3/users/76932fbe6ae54e1f9448459a32fe8268` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:07.356301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d7fb9-059c-4d6c-b2fa-4314ff09af88 None admin] RBAC: Authorizing `identity:get_user(user_id=76932fbe6ae54e1f9448459a32fe8268)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:07.357766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d7fb9-059c-4d6c-b2fa-4314ff09af88 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:07.359371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 590/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:39:07 2022] GET /identity/v3/users/76932fbe6ae54e1f9448459a32fe8268 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:07.365613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ac353-53a1-4f0e-80c6-9ed53c11005c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:07.399734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:07.461728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ac353-53a1-4f0e-80c6-9ed53c11005c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:07.495126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f1b73-c2e2-444a-bde9-3e94a6eec153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:07.495765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f1b73-c2e2-444a-bde9-3e94a6eec153 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:07.496048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f1b73-c2e2-444a-bde9-3e94a6eec153 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:07.496323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f1b73-c2e2-444a-bde9-3e94a6eec153 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:07.559872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f1b73-c2e2-444a-bde9-3e94a6eec153 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:07.561393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f1b73-c2e2-444a-bde9-3e94a6eec153 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:07.592259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:07.654765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 593/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:39:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:39:07.661145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7fb9-059c-4d6c-b2fa-4314ff09af88 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:07.747726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:07.755843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7fb9-059c-4d6c-b2fa-4314ff09af88 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:07.788112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d2ef0-4227-4a17-8242-6ce9a1ec8c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:07.789023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d2ef0-4227-4a17-8242-6ce9a1ec8c56 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:07.789500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d2ef0-4227-4a17-8242-6ce9a1ec8c56 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:07.790002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d2ef0-4227-4a17-8242-6ce9a1ec8c56 None admin] PATH_INFO: `/v3/projects/cc973a7659f64171b5aa99f142069e59` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:07.791567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d2ef0-4227-4a17-8242-6ce9a1ec8c56 None admin] RBAC: Authorizing `identity:get_project(project_id=cc973a7659f64171b5aa99f142069e59)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:07.793100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d2ef0-4227-4a17-8242-6ce9a1ec8c56 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:07.794791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 591/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:39:07 2022] GET /identity/v3/projects/cc973a7659f64171b5aa99f142069e59 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:07.801285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f1b73-c2e2-444a-bde9-3e94a6eec153 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:07.896230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f1b73-c2e2-444a-bde9-3e94a6eec153 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:07.928419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a1a80-33c8-4235-abe6-e31e694a4a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:07.929071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a1a80-33c8-4235-abe6-e31e694a4a6a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:07.929357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a1a80-33c8-4235-abe6-e31e694a4a6a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:07.929637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a1a80-33c8-4235-abe6-e31e694a4a6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:07.930468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a1a80-33c8-4235-abe6-e31e694a4a6a None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=cc973a7659f64171b5aa99f142069e59, user.id=76932fbe6ae54e1f9448459a32fe8268) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:39:07.930787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a1a80-33c8-4235-abe6-e31e694a4a6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:07.932240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a1a80-33c8-4235-abe6-e31e694a4a6a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:07.935037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:07.937845 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 594/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:39:07 2022] GET /identity/v3/role_assignments?user.id=76932fbe6ae54e1f9448459a32fe8268&scope.project.id=cc973a7659f64171b5aa99f142069e59&role.id=04c11dea81274a69974ad5c639f2c60b => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:10.064808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4016d6-b24d-4dcb-b9e4-b0dcb91065e3 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:10.065371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4016d6-b24d-4dcb-b9e4-b0dcb91065e3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:10.065371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4016d6-b24d-4dcb-b9e4-b0dcb91065e3 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:10.066708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 592/1186] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:10.071818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b1e60-b3fb-458f-96a9-bbe224025300 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:10.072189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b1e60-b3fb-458f-96a9-bbe224025300 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:10.072384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b1e60-b3fb-458f-96a9-bbe224025300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:10.089251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-c45b1e60-b3fb-458f-96a9-bbe224025300 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:10.090422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c45b1e60-b3fb-458f-96a9-bbe224025300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:10.158611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 595/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:39:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:39:10.180927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4016d6-b24d-4dcb-b9e4-b0dcb91065e3 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:10.278400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4016d6-b24d-4dcb-b9e4-b0dcb91065e3 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:10.311533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d97a63-b48c-4d68-8232-acc0e627dd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:10.312316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d97a63-b48c-4d68-8232-acc0e627dd48 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:10.312641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d97a63-b48c-4d68-8232-acc0e627dd48 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:10.312917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d97a63-b48c-4d68-8232-acc0e627dd48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:10.322836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d97a63-b48c-4d68-8232-acc0e627dd48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-f4d97a63-b48c-4d68-8232-acc0e627dd48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:39:10.330310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:39:10.331893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:39:10.332964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 593/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:39:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:39:10.338045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b1e60-b3fb-458f-96a9-bbe224025300 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:10.431539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b1e60-b3fb-458f-96a9-bbe224025300 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:10.459216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:10.464741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d20c3-7d1c-43d2-b238-8af8ef697410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:10.465394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d20c3-7d1c-43d2-b238-8af8ef697410 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:10.465676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d20c3-7d1c-43d2-b238-8af8ef697410 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:10.465949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d20c3-7d1c-43d2-b238-8af8ef697410 None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:10.466679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d20c3-7d1c-43d2-b238-8af8ef697410 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:39:10.466982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d20c3-7d1c-43d2-b238-8af8ef697410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:10.468338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d20c3-7d1c-43d2-b238-8af8ef697410 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:10.476658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 596/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:39:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:10.485403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d97a63-b48c-4d68-8232-acc0e627dd48 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:10.504479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:10.578371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d97a63-b48c-4d68-8232-acc0e627dd48 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:10.611368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f218c8-b67c-4aa4-944b-81a4680d45d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:10.612219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f218c8-b67c-4aa4-944b-81a4680d45d6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:10.612551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f218c8-b67c-4aa4-944b-81a4680d45d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:10.612830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f218c8-b67c-4aa4-944b-81a4680d45d6 None admin] PATH_INFO: `/v3/users/76932fbe6ae54e1f9448459a32fe8268` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:10.613990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f218c8-b67c-4aa4-944b-81a4680d45d6 None admin] RBAC: Authorizing `identity:get_user(user_id=76932fbe6ae54e1f9448459a32fe8268)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:10.615411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f218c8-b67c-4aa4-944b-81a4680d45d6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:10.617023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 594/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:39:10 2022] GET /identity/v3/users/76932fbe6ae54e1f9448459a32fe8268 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:10.622553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d20c3-7d1c-43d2-b238-8af8ef697410 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:10.719570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d20c3-7d1c-43d2-b238-8af8ef697410 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:10.751432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228aab2c-13c1-4b6b-9739-6687fa3a8675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:10.752274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228aab2c-13c1-4b6b-9739-6687fa3a8675 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:10.752586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228aab2c-13c1-4b6b-9739-6687fa3a8675 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:10.752867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228aab2c-13c1-4b6b-9739-6687fa3a8675 None admin] PATH_INFO: `/v3/projects/cc973a7659f64171b5aa99f142069e59` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:10.754573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228aab2c-13c1-4b6b-9739-6687fa3a8675 None admin] RBAC: Authorizing `identity:get_project(project_id=cc973a7659f64171b5aa99f142069e59)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:10.756010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228aab2c-13c1-4b6b-9739-6687fa3a8675 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:10.757701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 597/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:39:10 2022] GET /identity/v3/projects/cc973a7659f64171b5aa99f142069e59 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:10.763275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f218c8-b67c-4aa4-944b-81a4680d45d6 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:10.848507 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:10.859662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f218c8-b67c-4aa4-944b-81a4680d45d6 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:10.891817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfbb43-bbc3-4e98-8b8f-1ed2f940ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:10.892715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfbb43-bbc3-4e98-8b8f-1ed2f940ded1 None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:10.892996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfbb43-bbc3-4e98-8b8f-1ed2f940ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:10.893268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfbb43-bbc3-4e98-8b8f-1ed2f940ded1 None admin] PATH_INFO: `/v3/projects/cc973a7659f64171b5aa99f142069e59/users/76932fbe6ae54e1f9448459a32fe8268/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:10.895732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfbb43-bbc3-4e98-8b8f-1ed2f940ded1 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc973a7659f64171b5aa99f142069e59, user_id=76932fbe6ae54e1f9448459a32fe8268, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:10.897164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfbb43-bbc3-4e98-8b8f-1ed2f940ded1 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:10.995078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 595/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:39:10 2022] PUT /identity/v3/projects/cc973a7659f64171b5aa99f142069e59/users/76932fbe6ae54e1f9448459a32fe8268/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:39:13.154476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f959c0-ac16-4799-aa4c-89b622dee518 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:13.155018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f959c0-ac16-4799-aa4c-89b622dee518 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:13.155106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f959c0-ac16-4799-aa4c-89b622dee518 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:13.156452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 598/1193] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:39:13.161741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d62f7-eb1d-4b94-b98c-a06913f30d86 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:13.164104 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d62f7-eb1d-4b94-b98c-a06913f30d86 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:13.164104 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d62f7-eb1d-4b94-b98c-a06913f30d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:13.179513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-f11d62f7-eb1d-4b94-b98c-a06913f30d86 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:39:13.180727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f11d62f7-eb1d-4b94-b98c-a06913f30d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:39:13.244613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 596/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:39:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:39:13.268296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f959c0-ac16-4799-aa4c-89b622dee518 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:13.330329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:13.370687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f959c0-ac16-4799-aa4c-89b622dee518 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:13.403301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e346c0-f64d-447f-aff0-c431a03dbe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:13.404085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e346c0-f64d-447f-aff0-c431a03dbe0e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:13.404399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e346c0-f64d-447f-aff0-c431a03dbe0e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:13.404723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e346c0-f64d-447f-aff0-c431a03dbe0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:13.414446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e346c0-f64d-447f-aff0-c431a03dbe0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-d5e346c0-f64d-447f-aff0-c431a03dbe0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:39:13.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:39:13.423228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:39:13.424471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 599/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:39:13.429720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d62f7-eb1d-4b94-b98c-a06913f30d86 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:13.504332 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:13.524653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d62f7-eb1d-4b94-b98c-a06913f30d86 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:13.557954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6baa60-25e6-4073-b68a-73fab021e2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:13.558603 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6baa60-25e6-4073-b68a-73fab021e2c1 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:13.558883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6baa60-25e6-4073-b68a-73fab021e2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:13.559158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6baa60-25e6-4073-b68a-73fab021e2c1 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:13.559892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6baa60-25e6-4073-b68a-73fab021e2c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:39:13.560194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6baa60-25e6-4073-b68a-73fab021e2c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:13.561717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6baa60-25e6-4073-b68a-73fab021e2c1 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:13.569806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 597/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:13.575410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e346c0-f64d-447f-aff0-c431a03dbe0e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:13.671985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e346c0-f64d-447f-aff0-c431a03dbe0e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:13.691289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:13.704557 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a84dc-a857-4f58-bc0e-bdb6f6429d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:13.705403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a84dc-a857-4f58-bc0e-bdb6f6429d02 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:13.705685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a84dc-a857-4f58-bc0e-bdb6f6429d02 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:13.705959 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a84dc-a857-4f58-bc0e-bdb6f6429d02 None admin] PATH_INFO: `/v3/users/76932fbe6ae54e1f9448459a32fe8268` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:13.707150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a84dc-a857-4f58-bc0e-bdb6f6429d02 None admin] RBAC: Authorizing `identity:get_user(user_id=76932fbe6ae54e1f9448459a32fe8268)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:13.708618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a84dc-a857-4f58-bc0e-bdb6f6429d02 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:13.710199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 600/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Mon May 23 15:39:13 2022] GET /identity/v3/users/76932fbe6ae54e1f9448459a32fe8268 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:13.715665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6baa60-25e6-4073-b68a-73fab021e2c1 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:13.812813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6baa60-25e6-4073-b68a-73fab021e2c1 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:13.844990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595637e7-4fe4-4907-92e6-504e22436f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:13.845625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595637e7-4fe4-4907-92e6-504e22436f11 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:13.845909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595637e7-4fe4-4907-92e6-504e22436f11 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:13.846184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595637e7-4fe4-4907-92e6-504e22436f11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:13.856761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:13.911835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595637e7-4fe4-4907-92e6-504e22436f11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:13.913344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595637e7-4fe4-4907-92e6-504e22436f11 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:14.005352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 598/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:39:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:39:14.010997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a84dc-a857-4f58-bc0e-bdb6f6429d02 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:14.106387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a84dc-a857-4f58-bc0e-bdb6f6429d02 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:14.137515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3002c31-7eae-4d6f-a969-c0e24003a7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:14.138349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3002c31-7eae-4d6f-a969-c0e24003a7eb None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:14.138629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3002c31-7eae-4d6f-a969-c0e24003a7eb None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:14.138904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3002c31-7eae-4d6f-a969-c0e24003a7eb None admin] PATH_INFO: `/v3/projects/cc973a7659f64171b5aa99f142069e59` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:14.140048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3002c31-7eae-4d6f-a969-c0e24003a7eb None admin] RBAC: Authorizing `identity:get_project(project_id=cc973a7659f64171b5aa99f142069e59)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:14.141749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3002c31-7eae-4d6f-a969-c0e24003a7eb None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:14.143372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 601/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Mon May 23 15:39:14 2022] GET /identity/v3/projects/cc973a7659f64171b5aa99f142069e59 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:14.149687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595637e7-4fe4-4907-92e6-504e22436f11 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:39:14.189831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:14.245270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595637e7-4fe4-4907-92e6-504e22436f11 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:39:14.278765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274a9a0-4c32-42e7-9dc0-82ec5fe20af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:39:14.279348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274a9a0-4c32-42e7-9dc0-82ec5fe20af9 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:39:14.279630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274a9a0-4c32-42e7-9dc0-82ec5fe20af9 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:39:14.279902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274a9a0-4c32-42e7-9dc0-82ec5fe20af9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:39:14.280765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274a9a0-4c32-42e7-9dc0-82ec5fe20af9 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=cc973a7659f64171b5aa99f142069e59, user.id=76932fbe6ae54e1f9448459a32fe8268) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:39:14.281077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274a9a0-4c32-42e7-9dc0-82ec5fe20af9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:39:14.282509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274a9a0-4c32-42e7-9dc0-82ec5fe20af9 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:39:14.288549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 599/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:39:14 2022] GET /identity/v3/role_assignments?user.id=76932fbe6ae54e1f9448459a32fe8268&scope.project.id=cc973a7659f64171b5aa99f142069e59&role.id=04c11dea81274a69974ad5c639f2c60b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:39:24.133544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=72 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:39:24.285979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=48 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:29.653087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b9c47-b446-424f-bcdb-abc530e72951 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:29.653635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b9c47-b446-424f-bcdb-abc530e72951 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:29.653635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b9c47-b446-424f-bcdb-abc530e72951 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:29.654941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 602/1201] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:40:29.660053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b0772-04b7-4246-a69d-eb227b8090bd None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:29.661185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b0772-04b7-4246-a69d-eb227b8090bd None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:29.661185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b0772-04b7-4246-a69d-eb227b8090bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:29.677641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-770b0772-04b7-4246-a69d-eb227b8090bd None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:29.678808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-770b0772-04b7-4246-a69d-eb227b8090bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:29.742597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 600/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:40:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:40:29.766683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b9c47-b446-424f-bcdb-abc530e72951 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:29.868784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b9c47-b446-424f-bcdb-abc530e72951 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:29.902843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5fa9f-27cb-4c6c-8e82-45bbf64260dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:29.903678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5fa9f-27cb-4c6c-8e82-45bbf64260dd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:29.903915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5fa9f-27cb-4c6c-8e82-45bbf64260dd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:29.904202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5fa9f-27cb-4c6c-8e82-45bbf64260dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:29.913983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5fa9f-27cb-4c6c-8e82-45bbf64260dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:29.915421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5fa9f-27cb-4c6c-8e82-45bbf64260dd None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-6be5fa9f-27cb-4c6c-8e82-45bbf64260dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:29.932307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:29.938144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:29.939382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:29.939382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:29.939382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:29.939382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:29.939382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 603/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:29.939765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b0772-04b7-4246-a69d-eb227b8090bd None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:30.038345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b0772-04b7-4246-a69d-eb227b8090bd None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:30.065505 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:30.071214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c5a53-f50d-4556-8cb5-865f296081b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:30.072450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c5a53-f50d-4556-8cb5-865f296081b4 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:30.072755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c5a53-f50d-4556-8cb5-865f296081b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:30.073042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c5a53-f50d-4556-8cb5-865f296081b4 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:30.073761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c5a53-f50d-4556-8cb5-865f296081b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:30.074068 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c5a53-f50d-4556-8cb5-865f296081b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:30.075426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c5a53-f50d-4556-8cb5-865f296081b4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:30.086427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 601/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:30.092789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5fa9f-27cb-4c6c-8e82-45bbf64260dd None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:30.144136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:30.189548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5fa9f-27cb-4c6c-8e82-45bbf64260dd None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:30.222217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfe346-378e-49ad-a557-e6c894058a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:30.222881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfe346-378e-49ad-a557-e6c894058a4e None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:30.223165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfe346-378e-49ad-a557-e6c894058a4e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:30.223443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfe346-378e-49ad-a557-e6c894058a4e None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:30.224149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfe346-378e-49ad-a557-e6c894058a4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:30.225629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfe346-378e-49ad-a557-e6c894058a4e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:30.355893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 604/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Mon May 23 15:40:30 2022] POST /identity/v3/users => generated 251 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:40:32.574392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5fc69-3f20-4356-9696-fee6b06b0279 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:32.574898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5fc69-3f20-4356-9696-fee6b06b0279 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:32.574999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5fc69-3f20-4356-9696-fee6b06b0279 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:32.576344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 602/1206] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:40:32.581491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6dd9a4-a374-4d87-a08a-289ae1f8a8ce None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:32.581976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6dd9a4-a374-4d87-a08a-289ae1f8a8ce None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:32.582075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6dd9a4-a374-4d87-a08a-289ae1f8a8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:32.599222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-eb6dd9a4-a374-4d87-a08a-289ae1f8a8ce None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:32.600556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-eb6dd9a4-a374-4d87-a08a-289ae1f8a8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:32.665868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 605/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:40:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:40:32.689199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5fc69-3f20-4356-9696-fee6b06b0279 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:32.789052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5fc69-3f20-4356-9696-fee6b06b0279 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:32.823092 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b51439-402c-419e-a240-89328c05ef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:32.823877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b51439-402c-419e-a240-89328c05ef1e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:32.824156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b51439-402c-419e-a240-89328c05ef1e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:32.824508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b51439-402c-419e-a240-89328c05ef1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:32.834286 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b51439-402c-419e-a240-89328c05ef1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-20b51439-402c-419e-a240-89328c05ef1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:40:32.841867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:32.844806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:32.845869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 603/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:40:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:32.849045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6dd9a4-a374-4d87-a08a-289ae1f8a8ce None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:32.888301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:32.946868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6dd9a4-a374-4d87-a08a-289ae1f8a8ce None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:32.979604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444ab34-8b9b-4813-8fe6-5a272081b78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:32.980266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444ab34-8b9b-4813-8fe6-5a272081b78c None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:32.980575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444ab34-8b9b-4813-8fe6-5a272081b78c None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:32.980960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444ab34-8b9b-4813-8fe6-5a272081b78c None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:32.981711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444ab34-8b9b-4813-8fe6-5a272081b78c None admin] RBAC: Adding query filter params (name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:32.982011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444ab34-8b9b-4813-8fe6-5a272081b78c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:32.983380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444ab34-8b9b-4813-8fe6-5a272081b78c None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:32.991025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 606/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:40:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:32.997344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b51439-402c-419e-a240-89328c05ef1e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:33.047398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:33.096152 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b51439-402c-419e-a240-89328c05ef1e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:33.128605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e5b57-25f8-4687-8503-c1a4e0cd7811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:33.129383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e5b57-25f8-4687-8503-c1a4e0cd7811 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.129668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e5b57-25f8-4687-8503-c1a4e0cd7811 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.130191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e5b57-25f8-4687-8503-c1a4e0cd7811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.142466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e5b57-25f8-4687-8503-c1a4e0cd7811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:33.143912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e5b57-25f8-4687-8503-c1a4e0cd7811 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-bf4e5b57-25f8-4687-8503-c1a4e0cd7811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:33.160813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:33.161739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:33.162680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:33.162680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:33.162680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:33.162680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:33.162680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 604/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:33.168388 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444ab34-8b9b-4813-8fe6-5a272081b78c None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:33.263023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:33.265865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444ab34-8b9b-4813-8fe6-5a272081b78c None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:33.299677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fabb5d-88e0-47e9-95c1-c90f45c1bee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:33.302024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fabb5d-88e0-47e9-95c1-c90f45c1bee3 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.302024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fabb5d-88e0-47e9-95c1-c90f45c1bee3 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.302024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fabb5d-88e0-47e9-95c1-c90f45c1bee3 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.302024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fabb5d-88e0-47e9-95c1-c90f45c1bee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:33.302024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fabb5d-88e0-47e9-95c1-c90f45c1bee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:33.303470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fabb5d-88e0-47e9-95c1-c90f45c1bee3 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:33.314466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 607/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:33.320044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e5b57-25f8-4687-8503-c1a4e0cd7811 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:33.420125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e5b57-25f8-4687-8503-c1a4e0cd7811 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:33.421533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:33.454347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6745018-f34b-47dc-85f6-271a6c0dd0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:33.463524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6745018-f34b-47dc-85f6-271a6c0dd0fe None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.463524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6745018-f34b-47dc-85f6-271a6c0dd0fe None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.463524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6745018-f34b-47dc-85f6-271a6c0dd0fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.463524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6745018-f34b-47dc-85f6-271a6c0dd0fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:33.464647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6745018-f34b-47dc-85f6-271a6c0dd0fe None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-f6745018-f34b-47dc-85f6-271a6c0dd0fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:40:33.479850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:33.481005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:33.481993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 605/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:33.486786 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fabb5d-88e0-47e9-95c1-c90f45c1bee3 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:33.585980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fabb5d-88e0-47e9-95c1-c90f45c1bee3 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:33.620770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275cb89-8f27-48f8-831f-e455faebe88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:33.621434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275cb89-8f27-48f8-831f-e455faebe88d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.621714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275cb89-8f27-48f8-831f-e455faebe88d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.621988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275cb89-8f27-48f8-831f-e455faebe88d None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.622809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275cb89-8f27-48f8-831f-e455faebe88d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:33.623116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275cb89-8f27-48f8-831f-e455faebe88d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:33.624813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275cb89-8f27-48f8-831f-e455faebe88d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:33.642747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 608/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 23 15:40:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:33.648649 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6745018-f34b-47dc-85f6-271a6c0dd0fe None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:33.746995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6745018-f34b-47dc-85f6-271a6c0dd0fe None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:33.781549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db40e40-87c0-450d-97d0-fe81089a2417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:33.782194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db40e40-87c0-450d-97d0-fe81089a2417 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:33.782482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db40e40-87c0-450d-97d0-fe81089a2417 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:33.782762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db40e40-87c0-450d-97d0-fe81089a2417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:33.811096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:33.848258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db40e40-87c0-450d-97d0-fe81089a2417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:33.849734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db40e40-87c0-450d-97d0-fe81089a2417 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:33.945886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 606/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:40:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:40:33.952175 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275cb89-8f27-48f8-831f-e455faebe88d None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:33.971448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:34.050131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275cb89-8f27-48f8-831f-e455faebe88d None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:34.083663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693733d-6628-424a-a105-1bf0866ff6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:34.084474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693733d-6628-424a-a105-1bf0866ff6cb None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.084762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693733d-6628-424a-a105-1bf0866ff6cb None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.085040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693733d-6628-424a-a105-1bf0866ff6cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.094775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693733d-6628-424a-a105-1bf0866ff6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:34.096214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693733d-6628-424a-a105-1bf0866ff6cb None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-8693733d-6628-424a-a105-1bf0866ff6cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:34.113931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:34.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:34.119953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:34.119953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:34.119953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:34.119953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:34.119953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 609/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:34.121278 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db40e40-87c0-450d-97d0-fe81089a2417 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:34.181056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:34.219631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db40e40-87c0-450d-97d0-fe81089a2417 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:34.251768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2882291-df22-4aca-8b0a-13ada8b84483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:34.252450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2882291-df22-4aca-8b0a-13ada8b84483 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.252736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2882291-df22-4aca-8b0a-13ada8b84483 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.253012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2882291-df22-4aca-8b0a-13ada8b84483 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.253731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2882291-df22-4aca-8b0a-13ada8b84483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:34.254038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2882291-df22-4aca-8b0a-13ada8b84483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:34.255389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2882291-df22-4aca-8b0a-13ada8b84483 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.267353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 607/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:34.273279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693733d-6628-424a-a105-1bf0866ff6cb None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:34.343922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:34.375687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693733d-6628-424a-a105-1bf0866ff6cb None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:34.409779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f046b-389f-4b09-a6c0-2a519f398303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:34.410563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f046b-389f-4b09-a6c0-2a519f398303 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.410844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f046b-389f-4b09-a6c0-2a519f398303 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.411116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f046b-389f-4b09-a6c0-2a519f398303 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.417674 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f046b-389f-4b09-a6c0-2a519f398303 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:34.419166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f046b-389f-4b09-a6c0-2a519f398303 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-775f046b-389f-4b09-a6c0-2a519f398303 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:40:34.433167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:34.434201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:34.435510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:40:34.435510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:40:34.435510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:40:34.435510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:40:34.435510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:40:34.435510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:40:34.435510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:34.435510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:34.435510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 610/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:40:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:34.440099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2882291-df22-4aca-8b0a-13ada8b84483 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:34.539517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2882291-df22-4aca-8b0a-13ada8b84483 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:34.561317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:34.573238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd833b-779c-48c5-a860-e2754bf7d422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:34.573889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd833b-779c-48c5-a860-e2754bf7d422 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.574167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd833b-779c-48c5-a860-e2754bf7d422 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.574479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd833b-779c-48c5-a860-e2754bf7d422 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.575250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd833b-779c-48c5-a860-e2754bf7d422 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:34.575555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd833b-779c-48c5-a860-e2754bf7d422 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:34.576998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbd833b-779c-48c5-a860-e2754bf7d422 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.588757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 608/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:40:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:34.595280 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f046b-389f-4b09-a6c0-2a519f398303 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:34.692429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f046b-389f-4b09-a6c0-2a519f398303 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:34.720784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:34.726371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb9d6a-b8be-49f4-841b-bed1b654dcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:34.727026 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb9d6a-b8be-49f4-841b-bed1b654dcee None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:34.727317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb9d6a-b8be-49f4-841b-bed1b654dcee None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:34.727599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb9d6a-b8be-49f4-841b-bed1b654dcee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:34.728488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb9d6a-b8be-49f4-841b-bed1b654dcee None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=312ed2b555424c6a96b3d972ffea5ac0) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:34.728805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb9d6a-b8be-49f4-841b-bed1b654dcee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:34.730238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb9d6a-b8be-49f4-841b-bed1b654dcee None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:34.735511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 611/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:40:34 2022] GET /identity/v3/role_assignments?user.id=312ed2b555424c6a96b3d972ffea5ac0&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:36.892640 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bcb7c-7f6e-4c48-a1d0-5c4e9db68aee None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:36.893244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bcb7c-7f6e-4c48-a1d0-5c4e9db68aee None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:36.893244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bcb7c-7f6e-4c48-a1d0-5c4e9db68aee None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:36.894573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 609/1220] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:40:36.900002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a5a0-00bf-475a-8648-c4326a0c3a73 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:36.901067 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a5a0-00bf-475a-8648-c4326a0c3a73 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:36.901168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a5a0-00bf-475a-8648-c4326a0c3a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:36.917534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-8966a5a0-00bf-475a-8648-c4326a0c3a73 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:36.918703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8966a5a0-00bf-475a-8648-c4326a0c3a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:36.982906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 612/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:40:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:40:37.005043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bcb7c-7f6e-4c48-a1d0-5c4e9db68aee None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:37.105860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bcb7c-7f6e-4c48-a1d0-5c4e9db68aee None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:37.139765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8606d-8405-42ca-837c-cb33db967255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:37.140575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8606d-8405-42ca-837c-cb33db967255 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:37.140857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8606d-8405-42ca-837c-cb33db967255 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:37.141234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8606d-8405-42ca-837c-cb33db967255 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:37.151479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8606d-8405-42ca-837c-cb33db967255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-b1b8606d-8405-42ca-837c-cb33db967255 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:40:37.159158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:37.160202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:37.161174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 610/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:37.166861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966a5a0-00bf-475a-8648-c4326a0c3a73 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:37.263571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966a5a0-00bf-475a-8648-c4326a0c3a73 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:37.288074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:37.298629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef93a4-41a7-4473-bc68-5d8ec9f2486b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:37.299699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef93a4-41a7-4473-bc68-5d8ec9f2486b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:37.299699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef93a4-41a7-4473-bc68-5d8ec9f2486b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:37.299945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef93a4-41a7-4473-bc68-5d8ec9f2486b None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:37.300615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef93a4-41a7-4473-bc68-5d8ec9f2486b None admin] RBAC: Adding query filter params (name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:37.300929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef93a4-41a7-4473-bc68-5d8ec9f2486b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:37.302314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef93a4-41a7-4473-bc68-5d8ec9f2486b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:37.310619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 613/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:40:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:37.316346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8606d-8405-42ca-837c-cb33db967255 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:37.359057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:37.414568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8606d-8405-42ca-837c-cb33db967255 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:37.448737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ff46f-633f-463d-b876-0c8011af8b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:37.449582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ff46f-633f-463d-b876-0c8011af8b3a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:37.449869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ff46f-633f-463d-b876-0c8011af8b3a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:37.450149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ff46f-633f-463d-b876-0c8011af8b3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:37.460648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ff46f-633f-463d-b876-0c8011af8b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:37.462083 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ff46f-633f-463d-b876-0c8011af8b3a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-ed7ff46f-633f-463d-b876-0c8011af8b3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:37.479212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:37.480304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:37.481522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:37.481522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:37.481522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:37.481522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:37.481522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 611/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:37.486210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef93a4-41a7-4473-bc68-5d8ec9f2486b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:37.585029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef93a4-41a7-4473-bc68-5d8ec9f2486b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:37.618232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063fe52-7b78-4d23-b1f5-3d4370821219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:37.618897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063fe52-7b78-4d23-b1f5-3d4370821219 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:37.619756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063fe52-7b78-4d23-b1f5-3d4370821219 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:37.620036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063fe52-7b78-4d23-b1f5-3d4370821219 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:37.620793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063fe52-7b78-4d23-b1f5-3d4370821219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:37.621106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063fe52-7b78-4d23-b1f5-3d4370821219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:37.622486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063fe52-7b78-4d23-b1f5-3d4370821219 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:37.634855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 614/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:37.640685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ff46f-633f-463d-b876-0c8011af8b3a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:37.734023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:37.740542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ff46f-633f-463d-b876-0c8011af8b3a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:37.773547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8642c4-05f0-4da7-8277-ece2307421fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:37.774348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8642c4-05f0-4da7-8277-ece2307421fc None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:37.774639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8642c4-05f0-4da7-8277-ece2307421fc None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:37.774914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8642c4-05f0-4da7-8277-ece2307421fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:37.782383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8642c4-05f0-4da7-8277-ece2307421fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:37.783820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8642c4-05f0-4da7-8277-ece2307421fc None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-5d8642c4-05f0-4da7-8277-ece2307421fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:40:37.798651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:37.799616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:37.800671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 612/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:37.805651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063fe52-7b78-4d23-b1f5-3d4370821219 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:37.851402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:37.904171 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063fe52-7b78-4d23-b1f5-3d4370821219 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:37.936638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7324952-656b-485c-9e89-d0829e28ff63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:37.937297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324952-656b-485c-9e89-d0829e28ff63 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:37.937583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324952-656b-485c-9e89-d0829e28ff63 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:37.937862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324952-656b-485c-9e89-d0829e28ff63 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:37.938692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7324952-656b-485c-9e89-d0829e28ff63 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:37.939004 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7324952-656b-485c-9e89-d0829e28ff63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:37.940454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7324952-656b-485c-9e89-d0829e28ff63 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:37.957350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 615/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 23 15:40:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:37.963118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8642c4-05f0-4da7-8277-ece2307421fc None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.061321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8642c4-05f0-4da7-8277-ece2307421fc None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.094149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6a094-26a0-44cc-b7d9-35295a7b8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.094925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6a094-26a0-44cc-b7d9-35295a7b8adc None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.095208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6a094-26a0-44cc-b7d9-35295a7b8adc None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.095485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6a094-26a0-44cc-b7d9-35295a7b8adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.105597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6a094-26a0-44cc-b7d9-35295a7b8adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.107047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6a094-26a0-44cc-b7d9-35295a7b8adc None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-66c6a094-26a0-44cc-b7d9-35295a7b8adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:38.124810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:38.125889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:38.127201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:38.127201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:38.127201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:38.127201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:38.127201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 613/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:38.131696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7324952-656b-485c-9e89-d0829e28ff63 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.229501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7324952-656b-485c-9e89-d0829e28ff63 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.231792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:38.262938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e7728-8930-49e0-8d64-6e7d2f959fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.263600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e7728-8930-49e0-8d64-6e7d2f959fa3 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.263890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e7728-8930-49e0-8d64-6e7d2f959fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.264172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e7728-8930-49e0-8d64-6e7d2f959fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.264924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e7728-8930-49e0-8d64-6e7d2f959fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:38.265253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e7728-8930-49e0-8d64-6e7d2f959fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.266600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e7728-8930-49e0-8d64-6e7d2f959fa3 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.277925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 616/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:38.283773 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6a094-26a0-44cc-b7d9-35295a7b8adc None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.293149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:38.382501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6a094-26a0-44cc-b7d9-35295a7b8adc None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.416774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359c2a4-120d-40e1-a206-be8e42ca08af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.417831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359c2a4-120d-40e1-a206-be8e42ca08af None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.417831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359c2a4-120d-40e1-a206-be8e42ca08af None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.418161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359c2a4-120d-40e1-a206-be8e42ca08af None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.424727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359c2a4-120d-40e1-a206-be8e42ca08af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.426162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359c2a4-120d-40e1-a206-be8e42ca08af None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-d359c2a4-120d-40e1-a206-be8e42ca08af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:40:38.440395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:38.453497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:38.454613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:40:38.454613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:40:38.454613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:40:38.454613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:40:38.454613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:40:38.454613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:40:38.454613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:38.454613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:38.454613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 614/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:38.455249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e7728-8930-49e0-8d64-6e7d2f959fa3 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.545491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e7728-8930-49e0-8d64-6e7d2f959fa3 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.578898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49807d16-f25a-4050-a0e6-2a228d825aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.579560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49807d16-f25a-4050-a0e6-2a228d825aaa None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.579845 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49807d16-f25a-4050-a0e6-2a228d825aaa None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.580125 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49807d16-f25a-4050-a0e6-2a228d825aaa None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.580935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49807d16-f25a-4050-a0e6-2a228d825aaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:38.581249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49807d16-f25a-4050-a0e6-2a228d825aaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.582735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49807d16-f25a-4050-a0e6-2a228d825aaa None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.594830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 617/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:40:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:38.600780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359c2a4-120d-40e1-a206-be8e42ca08af None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:38.665086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:38.698859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359c2a4-120d-40e1-a206-be8e42ca08af None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:38.732038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf50630-6342-47f1-bc84-b33d0f387c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:38.735870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf50630-6342-47f1-bc84-b33d0f387c91 None admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:38.735870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf50630-6342-47f1-bc84-b33d0f387c91 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:38.735870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf50630-6342-47f1-bc84-b33d0f387c91 None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/312ed2b555424c6a96b3d972ffea5ac0/roles/527cc946916f4976ac459aae21d85772` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:38.747040 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf50630-6342-47f1-bc84-b33d0f387c91 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=312ed2b555424c6a96b3d972ffea5ac0, role_id=527cc946916f4976ac459aae21d85772)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:38.748593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf50630-6342-47f1-bc84-b33d0f387c91 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:38.804301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 615/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:40:38 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/312ed2b555424c6a96b3d972ffea5ac0/roles/527cc946916f4976ac459aae21d85772 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:40:40.977253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a2aa8-985f-4d41-99cf-622ef075d35c None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:40.977805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a2aa8-985f-4d41-99cf-622ef075d35c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:40.977805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a2aa8-985f-4d41-99cf-622ef075d35c None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:40.979791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 618/1233] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:40:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:40:40.984943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbad669-a05f-497d-ba1d-6aad60691836 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:40.985382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbad669-a05f-497d-ba1d-6aad60691836 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:40.985492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbad669-a05f-497d-ba1d-6aad60691836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:41.002787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-dbbad669-a05f-497d-ba1d-6aad60691836 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:41.006546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dbbad669-a05f-497d-ba1d-6aad60691836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:41.069617 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 616/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:40:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 15:40:41.093442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a2aa8-985f-4d41-99cf-622ef075d35c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:41.117915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:41.192938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a2aa8-985f-4d41-99cf-622ef075d35c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:41.225949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84abc7-e703-4556-8ede-373998fe3874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:41.226745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84abc7-e703-4556-8ede-373998fe3874 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:41.227024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84abc7-e703-4556-8ede-373998fe3874 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:41.232452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84abc7-e703-4556-8ede-373998fe3874 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:41.239721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84abc7-e703-4556-8ede-373998fe3874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ca84abc7-e703-4556-8ede-373998fe3874 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:40:41.248165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:41.249138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:41.250137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 619/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Mon May 23 15:40:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:41.255629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbad669-a05f-497d-ba1d-6aad60691836 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:41.307972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:41.360862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbad669-a05f-497d-ba1d-6aad60691836 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:41.393886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d2597-c697-48f1-a14e-ad19132a2db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:41.394555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d2597-c697-48f1-a14e-ad19132a2db7 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:41.394832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d2597-c697-48f1-a14e-ad19132a2db7 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:41.395359 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d2597-c697-48f1-a14e-ad19132a2db7 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:41.396105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d2597-c697-48f1-a14e-ad19132a2db7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:41.396434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d2597-c697-48f1-a14e-ad19132a2db7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:41.397779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d2597-c697-48f1-a14e-ad19132a2db7 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:41.407191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 617/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Mon May 23 15:40:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:40:41.412987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84abc7-e703-4556-8ede-373998fe3874 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:41.502598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:41.518011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84abc7-e703-4556-8ede-373998fe3874 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:41.551373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeadad-c56b-43a2-b9ed-4b91788378ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:41.552186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeadad-c56b-43a2-b9ed-4b91788378ec None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:41.552501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeadad-c56b-43a2-b9ed-4b91788378ec None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:41.552782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeadad-c56b-43a2-b9ed-4b91788378ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:41.564418 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeadad-c56b-43a2-b9ed-4b91788378ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:41.565958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeadad-c56b-43a2-b9ed-4b91788378ec None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-eadeadad-c56b-43a2-b9ed-4b91788378ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:41.583713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:41.585383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:41.586271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:41.586271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:41.586271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:41.586271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:41.586271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 620/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:40:41.590880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d2597-c697-48f1-a14e-ad19132a2db7 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:41.691625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d2597-c697-48f1-a14e-ad19132a2db7 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:41.700442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:41.724381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45d816-84b1-40dd-b107-835383e89cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:41.725113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45d816-84b1-40dd-b107-835383e89cb7 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:41.725397 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45d816-84b1-40dd-b107-835383e89cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:41.725678 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45d816-84b1-40dd-b107-835383e89cb7 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:41.726402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45d816-84b1-40dd-b107-835383e89cb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:41.726710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45d816-84b1-40dd-b107-835383e89cb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:41.728062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45d816-84b1-40dd-b107-835383e89cb7 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:41.739880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 618/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:41.745813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeadad-c56b-43a2-b9ed-4b91788378ec None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:41.842256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeadad-c56b-43a2-b9ed-4b91788378ec None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:41.875802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b92d8-3fd9-47c2-a284-82e750f7e28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:41.876634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b92d8-3fd9-47c2-a284-82e750f7e28b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:41.876916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b92d8-3fd9-47c2-a284-82e750f7e28b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:41.877192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b92d8-3fd9-47c2-a284-82e750f7e28b None admin] PATH_INFO: `/v3/users/placement` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:41.879865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:41.884796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b92d8-3fd9-47c2-a284-82e750f7e28b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:41.886829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b92d8-3fd9-47c2-a284-82e750f7e28b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-477b92d8-3fd9-47c2-a284-82e750f7e28b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:40:41.901412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:41.902580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:41.903748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 621/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:41.908307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45d816-84b1-40dd-b107-835383e89cb7 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:42.006511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45d816-84b1-40dd-b107-835383e89cb7 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:42.039582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7518b69-5051-450a-97db-7e4829f3ede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:42.040291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7518b69-5051-450a-97db-7e4829f3ede5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:42.040629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7518b69-5051-450a-97db-7e4829f3ede5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:42.040908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7518b69-5051-450a-97db-7e4829f3ede5 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:42.042233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7518b69-5051-450a-97db-7e4829f3ede5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:42.042540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7518b69-5051-450a-97db-7e4829f3ede5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:42.043985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7518b69-5051-450a-97db-7e4829f3ede5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:42.061362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 619/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 23 15:40:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:42.067164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b92d8-3fd9-47c2-a284-82e750f7e28b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:42.167117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b92d8-3fd9-47c2-a284-82e750f7e28b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:42.201316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211d8c5-c053-4c65-8810-0de43804b523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:42.201967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211d8c5-c053-4c65-8810-0de43804b523 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:42.202261 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211d8c5-c053-4c65-8810-0de43804b523 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:42.202539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211d8c5-c053-4c65-8810-0de43804b523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:42.262328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:42.268893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211d8c5-c053-4c65-8810-0de43804b523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:42.270615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211d8c5-c053-4c65-8810-0de43804b523 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:42.367575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 622/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:40:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:40:42.374167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7518b69-5051-450a-97db-7e4829f3ede5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:42.401395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:42.471782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7518b69-5051-450a-97db-7e4829f3ede5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:42.504319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fce347-74d3-4a50-957f-b5ae3f33512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:42.505130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fce347-74d3-4a50-957f-b5ae3f33512e None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:42.505413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fce347-74d3-4a50-957f-b5ae3f33512e None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:42.505685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fce347-74d3-4a50-957f-b5ae3f33512e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:42.515976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fce347-74d3-4a50-957f-b5ae3f33512e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:42.517432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fce347-74d3-4a50-957f-b5ae3f33512e None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-64fce347-74d3-4a50-957f-b5ae3f33512e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:42.538329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:42.539442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:42.540627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:42.540627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:42.540627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:42.540627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:42.540627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 620/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:42.545800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211d8c5-c053-4c65-8810-0de43804b523 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:42.636646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:42.642126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211d8c5-c053-4c65-8810-0de43804b523 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:42.675453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bafd2-4523-45df-81d2-0b434f7f8137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:42.676107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bafd2-4523-45df-81d2-0b434f7f8137 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:42.676413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bafd2-4523-45df-81d2-0b434f7f8137 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:42.676690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bafd2-4523-45df-81d2-0b434f7f8137 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:42.677404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bafd2-4523-45df-81d2-0b434f7f8137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:42.677712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bafd2-4523-45df-81d2-0b434f7f8137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:42.679071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bafd2-4523-45df-81d2-0b434f7f8137 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:42.689829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 623/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:42.695367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fce347-74d3-4a50-957f-b5ae3f33512e None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:42.774983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:42.796073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fce347-74d3-4a50-957f-b5ae3f33512e None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:42.828620 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea51977-a0e5-41bb-a555-884a8acf3548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:42.829413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea51977-a0e5-41bb-a555-884a8acf3548 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:42.829698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea51977-a0e5-41bb-a555-884a8acf3548 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:42.829975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea51977-a0e5-41bb-a555-884a8acf3548 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:42.836423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea51977-a0e5-41bb-a555-884a8acf3548 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:42.837861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea51977-a0e5-41bb-a555-884a8acf3548 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-0ea51977-a0e5-41bb-a555-884a8acf3548 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:40:42.851753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:42.852845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:42.854015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:40:42.854015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:40:42.854015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:40:42.854015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:40:42.854015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:40:42.854015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:40:42.854015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:42.854015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:42.854015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 621/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:40:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:42.858905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bafd2-4523-45df-81d2-0b434f7f8137 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:42.955884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bafd2-4523-45df-81d2-0b434f7f8137 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:42.989275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd946986-e350-43e1-9c56-56340518cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:42.989930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946986-e350-43e1-9c56-56340518cc56 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:42.990211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946986-e350-43e1-9c56-56340518cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:42.990487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946986-e350-43e1-9c56-56340518cc56 None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:42.991263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd946986-e350-43e1-9c56-56340518cc56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:42.991570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd946986-e350-43e1-9c56-56340518cc56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:42.993195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd946986-e350-43e1-9c56-56340518cc56 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:43.005440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 624/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:43.011940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea51977-a0e5-41bb-a555-884a8acf3548 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:43.111189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea51977-a0e5-41bb-a555-884a8acf3548 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:43.143866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0af163-7afd-4619-9171-37d75da8ceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:43.144538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0af163-7afd-4619-9171-37d75da8ceda None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:43.144821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0af163-7afd-4619-9171-37d75da8ceda None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:43.145356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0af163-7afd-4619-9171-37d75da8ceda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:43.146171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0af163-7afd-4619-9171-37d75da8ceda None admin] RBAC: Adding query filter params (role.id=527cc946916f4976ac459aae21d85772, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=312ed2b555424c6a96b3d972ffea5ac0) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:43.146482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0af163-7afd-4619-9171-37d75da8ceda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:43.147896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0af163-7afd-4619-9171-37d75da8ceda None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:43.153613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 622/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:40:43 2022] GET /identity/v3/role_assignments?user.id=312ed2b555424c6a96b3d972ffea5ac0&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=527cc946916f4976ac459aae21d85772 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:45.385962 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8cfed-43b1-476c-b998-62f7f4d514de None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:45.386491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8cfed-43b1-476c-b998-62f7f4d514de None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:45.386582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8cfed-43b1-476c-b998-62f7f4d514de None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:45.387978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 625/1247] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:40:45.393208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4a968-36d0-4abf-9c6c-7207798abc86 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:45.393690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4a968-36d0-4abf-9c6c-7207798abc86 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:45.393785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4a968-36d0-4abf-9c6c-7207798abc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:45.399018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:45.411707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-f7b4a968-36d0-4abf-9c6c-7207798abc86 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:45.412929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f7b4a968-36d0-4abf-9c6c-7207798abc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:45.477974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 623/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:40:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:40:45.501890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8cfed-43b1-476c-b998-62f7f4d514de None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:45.519992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:45.601362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8cfed-43b1-476c-b998-62f7f4d514de None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:45.633852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5a848-281a-444a-8b7e-b33c4fa41b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:45.634626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5a848-281a-444a-8b7e-b33c4fa41b63 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:45.634907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5a848-281a-444a-8b7e-b33c4fa41b63 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:45.635706 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5a848-281a-444a-8b7e-b33c4fa41b63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:45.645828 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5a848-281a-444a-8b7e-b33c4fa41b63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-43d5a848-281a-444a-8b7e-b33c4fa41b63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:40:45.653947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:45.657248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:45.658123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 626/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:40:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:45.661202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4a968-36d0-4abf-9c6c-7207798abc86 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:45.759260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4a968-36d0-4abf-9c6c-7207798abc86 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:45.788943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:45.793679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac6dea-40ab-444c-929c-7178d2ea9e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:45.794342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac6dea-40ab-444c-929c-7178d2ea9e1c None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:45.794622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac6dea-40ab-444c-929c-7178d2ea9e1c None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:45.794890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac6dea-40ab-444c-929c-7178d2ea9e1c None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:45.795616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac6dea-40ab-444c-929c-7178d2ea9e1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:45.795916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac6dea-40ab-444c-929c-7178d2ea9e1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:45.797317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac6dea-40ab-444c-929c-7178d2ea9e1c None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:45.805079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 624/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:40:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:45.811089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5a848-281a-444a-8b7e-b33c4fa41b63 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:45.887349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:45.909896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5a848-281a-444a-8b7e-b33c4fa41b63 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:45.941745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cc8f9-326b-4c69-9300-25957487f10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:45.942522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cc8f9-326b-4c69-9300-25957487f10e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:45.942809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cc8f9-326b-4c69-9300-25957487f10e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:45.943088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cc8f9-326b-4c69-9300-25957487f10e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:45.953855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cc8f9-326b-4c69-9300-25957487f10e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:45.955302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cc8f9-326b-4c69-9300-25957487f10e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-115cc8f9-326b-4c69-9300-25957487f10e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:45.972721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:45.973767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:45.975012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:45.975012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:45.975012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:45.975012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:45.975012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 627/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:45.980220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac6dea-40ab-444c-929c-7178d2ea9e1c None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:46.082618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac6dea-40ab-444c-929c-7178d2ea9e1c None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:46.114644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f821ce6-2fdd-44ca-8022-e6f0f1598bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:46.115305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f821ce6-2fdd-44ca-8022-e6f0f1598bcf None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:46.115589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f821ce6-2fdd-44ca-8022-e6f0f1598bcf None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:46.115866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f821ce6-2fdd-44ca-8022-e6f0f1598bcf None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:46.116613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f821ce6-2fdd-44ca-8022-e6f0f1598bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:46.116922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f821ce6-2fdd-44ca-8022-e6f0f1598bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:46.118318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f821ce6-2fdd-44ca-8022-e6f0f1598bcf None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:46.130323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 625/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:46.136534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cc8f9-326b-4c69-9300-25957487f10e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:46.233843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cc8f9-326b-4c69-9300-25957487f10e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:46.258844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:46.266981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6160f4-e34d-4702-a839-e6fc4d9bc5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:46.267773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6160f4-e34d-4702-a839-e6fc4d9bc5d8 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:46.268054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6160f4-e34d-4702-a839-e6fc4d9bc5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:46.268335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6160f4-e34d-4702-a839-e6fc4d9bc5d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:46.275842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6160f4-e34d-4702-a839-e6fc4d9bc5d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:46.277341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6160f4-e34d-4702-a839-e6fc4d9bc5d8 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-fe6160f4-e34d-4702-a839-e6fc4d9bc5d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:40:46.292409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:46.293547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:46.294934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 628/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:46.299379 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f821ce6-2fdd-44ca-8022-e6f0f1598bcf None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:46.355065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:46.398277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f821ce6-2fdd-44ca-8022-e6f0f1598bcf None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:46.430438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c740b-877e-4a18-ba68-3d52709c3588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:46.431095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c740b-877e-4a18-ba68-3d52709c3588 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:46.431410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c740b-877e-4a18-ba68-3d52709c3588 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:46.431684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c740b-877e-4a18-ba68-3d52709c3588 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:46.433222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c740b-877e-4a18-ba68-3d52709c3588 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:46.433531 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c740b-877e-4a18-ba68-3d52709c3588 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:46.434937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c740b-877e-4a18-ba68-3d52709c3588 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:46.452610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 626/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 23 15:40:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:46.458304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6160f4-e34d-4702-a839-e6fc4d9bc5d8 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:46.558341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6160f4-e34d-4702-a839-e6fc4d9bc5d8 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:46.590742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb87da-d4e0-4e4e-adc0-fa440e5371a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:46.591388 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb87da-d4e0-4e4e-adc0-fa440e5371a0 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:46.591668 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb87da-d4e0-4e4e-adc0-fa440e5371a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:46.591972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb87da-d4e0-4e4e-adc0-fa440e5371a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:46.646306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:46.658129 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb87da-d4e0-4e4e-adc0-fa440e5371a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:46.659585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb87da-d4e0-4e4e-adc0-fa440e5371a0 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:46.753405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 629/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:40:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:40:46.759321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c740b-877e-4a18-ba68-3d52709c3588 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:46.857102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c740b-877e-4a18-ba68-3d52709c3588 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:46.870668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:46.890699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e519c-a4b4-4420-b8fe-ce3f699bdeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:46.891485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e519c-a4b4-4420-b8fe-ce3f699bdeec None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:46.891766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e519c-a4b4-4420-b8fe-ce3f699bdeec None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:46.892040 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e519c-a4b4-4420-b8fe-ce3f699bdeec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:46.902471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e519c-a4b4-4420-b8fe-ce3f699bdeec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:46.903897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e519c-a4b4-4420-b8fe-ce3f699bdeec None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-9a5e519c-a4b4-4420-b8fe-ce3f699bdeec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:46.921291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:46.922349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:46.923647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:46.923647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:46.923647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:46.923647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:46.923647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 627/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:46.928377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb87da-d4e0-4e4e-adc0-fa440e5371a0 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:47.014732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:47.027129 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb87da-d4e0-4e4e-adc0-fa440e5371a0 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:47.063396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd254ea0-bf23-4a4c-97f0-81577023c970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:47.064114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd254ea0-bf23-4a4c-97f0-81577023c970 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:47.064423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd254ea0-bf23-4a4c-97f0-81577023c970 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:47.064699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd254ea0-bf23-4a4c-97f0-81577023c970 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:47.065415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd254ea0-bf23-4a4c-97f0-81577023c970 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:47.065743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd254ea0-bf23-4a4c-97f0-81577023c970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:47.067120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd254ea0-bf23-4a4c-97f0-81577023c970 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:47.078460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 630/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:47.084257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e519c-a4b4-4420-b8fe-ce3f699bdeec None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:47.182871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e519c-a4b4-4420-b8fe-ce3f699bdeec None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:47.215715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7f68a-1686-4dc3-af06-2ea5e0b2ce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:47.216551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7f68a-1686-4dc3-af06-2ea5e0b2ce73 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:47.216835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7f68a-1686-4dc3-af06-2ea5e0b2ce73 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:47.217113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7f68a-1686-4dc3-af06-2ea5e0b2ce73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:47.223711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7f68a-1686-4dc3-af06-2ea5e0b2ce73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:47.225169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7f68a-1686-4dc3-af06-2ea5e0b2ce73 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-e2c7f68a-1686-4dc3-af06-2ea5e0b2ce73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:40:47.239067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:47.240158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:47.241355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:40:47.241355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:40:47.241355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:40:47.241355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:40:47.241355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:40:47.241355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:40:47.241355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:47.241355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:47.241355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 628/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:47.246997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd254ea0-bf23-4a4c-97f0-81577023c970 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:47.349370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd254ea0-bf23-4a4c-97f0-81577023c970 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:47.381443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d9eca-85e6-43a9-94d7-ac8944ea893d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:47.382126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d9eca-85e6-43a9-94d7-ac8944ea893d None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:47.382411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d9eca-85e6-43a9-94d7-ac8944ea893d None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:47.382692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d9eca-85e6-43a9-94d7-ac8944ea893d None admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:47.383472 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d9eca-85e6-43a9-94d7-ac8944ea893d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:47.383784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d9eca-85e6-43a9-94d7-ac8944ea893d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:47.385841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d9eca-85e6-43a9-94d7-ac8944ea893d None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:47.398116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 631/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:47.404826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7f68a-1686-4dc3-af06-2ea5e0b2ce73 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:47.430229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:47.502923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7f68a-1686-4dc3-af06-2ea5e0b2ce73 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:47.536908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67d8b8-ff56-4929-84ad-4b89d818ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:47.537557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67d8b8-ff56-4929-84ad-4b89d818ec79 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:47.538026 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67d8b8-ff56-4929-84ad-4b89d818ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:47.538186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67d8b8-ff56-4929-84ad-4b89d818ec79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:47.538918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67d8b8-ff56-4929-84ad-4b89d818ec79 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=312ed2b555424c6a96b3d972ffea5ac0) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:47.539232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67d8b8-ff56-4929-84ad-4b89d818ec79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:47.540683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67d8b8-ff56-4929-84ad-4b89d818ec79 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:47.546033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 629/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:40:47 2022] GET /identity/v3/role_assignments?user.id=312ed2b555424c6a96b3d972ffea5ac0&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=04c11dea81274a69974ad5c639f2c60b => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:49.690989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ecfcb-e404-426e-8e66-3c193fac8495 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:49.691512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ecfcb-e404-426e-8e66-3c193fac8495 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:49.691512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ecfcb-e404-426e-8e66-3c193fac8495 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:49.693071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 632/1261] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:40:49.699292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3e268-609c-4f18-a7c1-ca8fd2b7f2f6 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:49.699709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3e268-609c-4f18-a7c1-ca8fd2b7f2f6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:49.699817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3e268-609c-4f18-a7c1-ca8fd2b7f2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:49.716889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8dc3e268-609c-4f18-a7c1-ca8fd2b7f2f6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:49.718103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8dc3e268-609c-4f18-a7c1-ca8fd2b7f2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:49.784350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 630/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:40:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:40:49.807650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ecfcb-e404-426e-8e66-3c193fac8495 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:49.818341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:49.908385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ecfcb-e404-426e-8e66-3c193fac8495 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:49.941741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81fe52-96db-44dc-b451-9281128f1078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:49.942460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81fe52-96db-44dc-b451-9281128f1078 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:49.942741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81fe52-96db-44dc-b451-9281128f1078 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:49.943121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81fe52-96db-44dc-b451-9281128f1078 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:49.953782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81fe52-96db-44dc-b451-9281128f1078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-5a81fe52-96db-44dc-b451-9281128f1078 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:40:49.961649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:49.962851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:49.964489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 633/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:40:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:49.969629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3e268-609c-4f18-a7c1-ca8fd2b7f2f6 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:49.988499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:50.065149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3e268-609c-4f18-a7c1-ca8fd2b7f2f6 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:50.098869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ef114-1501-424b-a7b7-debea67a6445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:50.099536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ef114-1501-424b-a7b7-debea67a6445 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:50.099817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ef114-1501-424b-a7b7-debea67a6445 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:50.100090 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ef114-1501-424b-a7b7-debea67a6445 None admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:50.100867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ef114-1501-424b-a7b7-debea67a6445 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:50.101174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ef114-1501-424b-a7b7-debea67a6445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:50.102655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ef114-1501-424b-a7b7-debea67a6445 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:50.116057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 631/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:40:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:40:50.117302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81fe52-96db-44dc-b451-9281128f1078 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:50.185074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:50.218024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81fe52-96db-44dc-b451-9281128f1078 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:50.249434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ad15b-e7d2-4557-b24e-6a5c348fafff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:50.250223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ad15b-e7d2-4557-b24e-6a5c348fafff None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:50.250570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ad15b-e7d2-4557-b24e-6a5c348fafff None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:50.250847 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ad15b-e7d2-4557-b24e-6a5c348fafff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:50.261616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ad15b-e7d2-4557-b24e-6a5c348fafff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:50.263404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ad15b-e7d2-4557-b24e-6a5c348fafff None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-8d5ad15b-e7d2-4557-b24e-6a5c348fafff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:50.282092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:50.283055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:50.283988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:50.283988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:50.283988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:50.283988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:50.283988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 634/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:50.289303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ef114-1501-424b-a7b7-debea67a6445 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:50.369498 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:50.388303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ef114-1501-424b-a7b7-debea67a6445 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:50.421051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d02af1-f53b-43e4-bd73-5d7c2d8e3720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:50.421747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d02af1-f53b-43e4-bd73-5d7c2d8e3720 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:50.422036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d02af1-f53b-43e4-bd73-5d7c2d8e3720 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:50.422316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d02af1-f53b-43e4-bd73-5d7c2d8e3720 None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:50.423043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d02af1-f53b-43e4-bd73-5d7c2d8e3720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:50.423351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d02af1-f53b-43e4-bd73-5d7c2d8e3720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:50.424781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d02af1-f53b-43e4-bd73-5d7c2d8e3720 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:50.436937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 632/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:40:50.442660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ad15b-e7d2-4557-b24e-6a5c348fafff None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:50.540472 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ad15b-e7d2-4557-b24e-6a5c348fafff None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:50.560701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:50.574673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9845e5-66ff-43dd-9f1a-3aa6c270af6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:50.575469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9845e5-66ff-43dd-9f1a-3aa6c270af6a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:50.575747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9845e5-66ff-43dd-9f1a-3aa6c270af6a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:50.576117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9845e5-66ff-43dd-9f1a-3aa6c270af6a None admin] PATH_INFO: `/v3/users/placement` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:50.583605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9845e5-66ff-43dd-9f1a-3aa6c270af6a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:50.585076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9845e5-66ff-43dd-9f1a-3aa6c270af6a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-ab9845e5-66ff-43dd-9f1a-3aa6c270af6a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:40:50.599206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:50.600153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:50.601223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 635/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:50.606255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d02af1-f53b-43e4-bd73-5d7c2d8e3720 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:50.705151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d02af1-f53b-43e4-bd73-5d7c2d8e3720 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:50.737825 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944266c-ed7c-4fba-9235-cb759ad4aaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:50.738491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944266c-ed7c-4fba-9235-cb759ad4aaf5 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:50.738777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944266c-ed7c-4fba-9235-cb759ad4aaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:50.739054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944266c-ed7c-4fba-9235-cb759ad4aaf5 None admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:50.739891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944266c-ed7c-4fba-9235-cb759ad4aaf5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:50.740201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944266c-ed7c-4fba-9235-cb759ad4aaf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:50.741746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944266c-ed7c-4fba-9235-cb759ad4aaf5 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:50.750642 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:50.758915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 633/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 23 15:40:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:50.765229 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9845e5-66ff-43dd-9f1a-3aa6c270af6a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:50.864258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9845e5-66ff-43dd-9f1a-3aa6c270af6a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:50.896952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3699d51-06e8-4890-a936-c3330e2622bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:50.897744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699d51-06e8-4890-a936-c3330e2622bc None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:50.898033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699d51-06e8-4890-a936-c3330e2622bc None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:50.898312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699d51-06e8-4890-a936-c3330e2622bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:50.909036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3699d51-06e8-4890-a936-c3330e2622bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:50.910477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3699d51-06e8-4890-a936-c3330e2622bc None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-f3699d51-06e8-4890-a936-c3330e2622bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:50.926985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:50.928035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:50.929161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:50.929161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:50.929161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:50.929161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:50.929161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 636/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 15:40:50.933943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944266c-ed7c-4fba-9235-cb759ad4aaf5 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:51.033184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944266c-ed7c-4fba-9235-cb759ad4aaf5 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:51.066228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595e14f-46a6-446a-954c-be28fa7a535d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:51.066423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595e14f-46a6-446a-954c-be28fa7a535d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:51.066729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595e14f-46a6-446a-954c-be28fa7a535d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:51.067003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595e14f-46a6-446a-954c-be28fa7a535d None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:51.067867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595e14f-46a6-446a-954c-be28fa7a535d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:51.068172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595e14f-46a6-446a-954c-be28fa7a535d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:51.069560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595e14f-46a6-446a-954c-be28fa7a535d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:51.081430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 634/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:51.087242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3699d51-06e8-4890-a936-c3330e2622bc None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:51.120711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:51.188608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3699d51-06e8-4890-a936-c3330e2622bc None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:51.220653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad49443-cd85-44a1-a851-cd3eeb4906e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:51.221440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad49443-cd85-44a1-a851-cd3eeb4906e9 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:51.221893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad49443-cd85-44a1-a851-cd3eeb4906e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:51.222266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad49443-cd85-44a1-a851-cd3eeb4906e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:51.228613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad49443-cd85-44a1-a851-cd3eeb4906e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:51.230611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad49443-cd85-44a1-a851-cd3eeb4906e9 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-dad49443-cd85-44a1-a851-cd3eeb4906e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:40:51.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:51.250709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:51.251841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:40:51.251841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:40:51.251841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:40:51.251841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:40:51.251841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:40:51.251841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:40:51.251841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:51.251841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:51.251841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 637/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:40:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:51.253703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595e14f-46a6-446a-954c-be28fa7a535d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:51.313742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:51.353947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595e14f-46a6-446a-954c-be28fa7a535d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:51.386124 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd28088-29bc-4efc-8d76-7a69ec0315b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:51.386723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd28088-29bc-4efc-8d76-7a69ec0315b7 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:51.387006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd28088-29bc-4efc-8d76-7a69ec0315b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:51.387296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd28088-29bc-4efc-8d76-7a69ec0315b7 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:51.388088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd28088-29bc-4efc-8d76-7a69ec0315b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:51.388422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd28088-29bc-4efc-8d76-7a69ec0315b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:51.389841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd28088-29bc-4efc-8d76-7a69ec0315b7 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:51.402953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 635/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:40:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:51.408910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad49443-cd85-44a1-a851-cd3eeb4906e9 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:51.499764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:51.508242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad49443-cd85-44a1-a851-cd3eeb4906e9 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:51.542616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ac44d-efca-47ae-b17e-b860780fe5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:51.543508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ac44d-efca-47ae-b17e-b860780fe5f3 None admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:51.543792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ac44d-efca-47ae-b17e-b860780fe5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:51.544073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ac44d-efca-47ae-b17e-b860780fe5f3 None admin] PATH_INFO: `/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/312ed2b555424c6a96b3d972ffea5ac0/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:51.546052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ac44d-efca-47ae-b17e-b860780fe5f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b8dd1fc12a644f79405a3ac265147b5, user_id=312ed2b555424c6a96b3d972ffea5ac0, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:51.547478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ac44d-efca-47ae-b17e-b860780fe5f3 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:51.755377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 638/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Mon May 23 15:40:51 2022] PUT /identity/v3/projects/7b8dd1fc12a644f79405a3ac265147b5/users/312ed2b555424c6a96b3d972ffea5ac0/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:40:53.905654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4981be-77c1-43de-a285-ee4d9d9c09bf None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:53.906211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4981be-77c1-43de-a285-ee4d9d9c09bf None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:53.906211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4981be-77c1-43de-a285-ee4d9d9c09bf None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:53.907573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 636/1274] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:40:53.912900 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918af45-83e2-4969-ab24-00a39924e422 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:53.913327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918af45-83e2-4969-ab24-00a39924e422 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:53.913468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918af45-83e2-4969-ab24-00a39924e422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:53.930275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-3918af45-83e2-4969-ab24-00a39924e422 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:53.931471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3918af45-83e2-4969-ab24-00a39924e422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:53.996956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 639/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:40:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:40:54.020979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4981be-77c1-43de-a285-ee4d9d9c09bf None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:54.121632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4981be-77c1-43de-a285-ee4d9d9c09bf None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:54.138315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:54.154470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dabf97-9311-46ff-9dee-a3e771f26a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:54.155253 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dabf97-9311-46ff-9dee-a3e771f26a30 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:54.155536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dabf97-9311-46ff-9dee-a3e771f26a30 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:54.155863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dabf97-9311-46ff-9dee-a3e771f26a30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:54.165552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dabf97-9311-46ff-9dee-a3e771f26a30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-c6dabf97-9311-46ff-9dee-a3e771f26a30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 15:40:54.173574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:54.174677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:54.175715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 637/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Mon May 23 15:40:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:54.180873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918af45-83e2-4969-ab24-00a39924e422 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:54.259137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:54.276912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918af45-83e2-4969-ab24-00a39924e422 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:54.310274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64911f8-245f-4619-82f1-c339447a962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:54.310975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64911f8-245f-4619-82f1-c339447a962e None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:54.311228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64911f8-245f-4619-82f1-c339447a962e None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:54.311504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64911f8-245f-4619-82f1-c339447a962e None admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:54.312243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64911f8-245f-4619-82f1-c339447a962e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:54.312585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64911f8-245f-4619-82f1-c339447a962e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:54.313943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64911f8-245f-4619-82f1-c339447a962e None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:54.325080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 640/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Mon May 23 15:40:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:54.327757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dabf97-9311-46ff-9dee-a3e771f26a30 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:54.429419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dabf97-9311-46ff-9dee-a3e771f26a30 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:54.462596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e76b9-1153-4cd9-a2c4-6d36b34d4763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:54.463386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e76b9-1153-4cd9-a2c4-6d36b34d4763 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:54.464178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e76b9-1153-4cd9-a2c4-6d36b34d4763 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:54.464485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e76b9-1153-4cd9-a2c4-6d36b34d4763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:54.474819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e76b9-1153-4cd9-a2c4-6d36b34d4763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:54.476257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e76b9-1153-4cd9-a2c4-6d36b34d4763 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-740e76b9-1153-4cd9-a2c4-6d36b34d4763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:54.493605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:54.494664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:54.496031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:54.496031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:54.496031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:54.496031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:54.496031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 638/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:54.500520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64911f8-245f-4619-82f1-c339447a962e None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:54.597536 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64911f8-245f-4619-82f1-c339447a962e None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:54.629767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583379d-87cf-4302-b1af-3608cdc0a108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:54.630425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583379d-87cf-4302-b1af-3608cdc0a108 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:54.630718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583379d-87cf-4302-b1af-3608cdc0a108 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:54.631000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583379d-87cf-4302-b1af-3608cdc0a108 None admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:54.631718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583379d-87cf-4302-b1af-3608cdc0a108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:54.632030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583379d-87cf-4302-b1af-3608cdc0a108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:54.633491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583379d-87cf-4302-b1af-3608cdc0a108 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:54.645495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 641/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:54.651423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e76b9-1153-4cd9-a2c4-6d36b34d4763 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:54.681792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:54.749154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e76b9-1153-4cd9-a2c4-6d36b34d4763 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:54.783611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f4a39-ccf4-4940-a3b8-b37e927c60f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:54.784423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f4a39-ccf4-4940-a3b8-b37e927c60f7 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:54.784709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f4a39-ccf4-4940-a3b8-b37e927c60f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:54.784987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f4a39-ccf4-4940-a3b8-b37e927c60f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:54.791815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f4a39-ccf4-4940-a3b8-b37e927c60f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:54.793259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f4a39-ccf4-4940-a3b8-b37e927c60f7 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-926f4a39-ccf4-4940-a3b8-b37e927c60f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 15:40:54.807453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:54.808577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:54.809553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 639/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:54.814483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583379d-87cf-4302-b1af-3608cdc0a108 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:54.818278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:54.912712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583379d-87cf-4302-b1af-3608cdc0a108 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:54.944969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e0e62-48ce-46f0-b7af-9cecb2bbbe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:54.945573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0e62-48ce-46f0-b7af-9cecb2bbbe31 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:54.945855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0e62-48ce-46f0-b7af-9cecb2bbbe31 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:54.946130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0e62-48ce-46f0-b7af-9cecb2bbbe31 None admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:54.946986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e0e62-48ce-46f0-b7af-9cecb2bbbe31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:54.947329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e0e62-48ce-46f0-b7af-9cecb2bbbe31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:54.949289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e0e62-48ce-46f0-b7af-9cecb2bbbe31 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:54.967655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 642/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Mon May 23 15:40:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:54.973398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f4a39-ccf4-4940-a3b8-b37e927c60f7 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:55.062109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:55.071207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f4a39-ccf4-4940-a3b8-b37e927c60f7 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:55.104984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8404ed-19c4-4840-8698-592fac818c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:55.105648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8404ed-19c4-4840-8698-592fac818c84 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:55.105929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8404ed-19c4-4840-8698-592fac818c84 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:55.106203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8404ed-19c4-4840-8698-592fac818c84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:55.172354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8404ed-19c4-4840-8698-592fac818c84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:55.173803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8404ed-19c4-4840-8698-592fac818c84 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:55.259802 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:55.270430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 640/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Mon May 23 15:40:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 15:40:55.276459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e0e62-48ce-46f0-b7af-9cecb2bbbe31 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:55.336223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:55.373523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e0e62-48ce-46f0-b7af-9cecb2bbbe31 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:55.406734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45482afe-3c45-4476-be70-26b2d40d738b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:55.407519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45482afe-3c45-4476-be70-26b2d40d738b None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:55.407801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45482afe-3c45-4476-be70-26b2d40d738b None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:55.408076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45482afe-3c45-4476-be70-26b2d40d738b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:55.418939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45482afe-3c45-4476-be70-26b2d40d738b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:55.420379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45482afe-3c45-4476-be70-26b2d40d738b None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-45482afe-3c45-4476-be70-26b2d40d738b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 15:40:55.436954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:55.438011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:55.438979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 15:40:55.438979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 15:40:55.438979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 15:40:55.438979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:55.438979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 643/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:55.443999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8404ed-19c4-4840-8698-592fac818c84 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:55.543386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8404ed-19c4-4840-8698-592fac818c84 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:55.576021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6a4f1-cc95-4264-b113-c5ea2aa90f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:55.576764 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6a4f1-cc95-4264-b113-c5ea2aa90f7a None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:55.577037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6a4f1-cc95-4264-b113-c5ea2aa90f7a None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:55.577312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6a4f1-cc95-4264-b113-c5ea2aa90f7a None admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:55.578029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6a4f1-cc95-4264-b113-c5ea2aa90f7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:55.578334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6a4f1-cc95-4264-b113-c5ea2aa90f7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:55.579673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6a4f1-cc95-4264-b113-c5ea2aa90f7a None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:55.591664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 641/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Mon May 23 15:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:55.597294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45482afe-3c45-4476-be70-26b2d40d738b None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:55.695584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45482afe-3c45-4476-be70-26b2d40d738b None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:55.710227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:55.733477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3fb35-8025-4ebf-99c3-49e7705ecb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:55.734299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3fb35-8025-4ebf-99c3-49e7705ecb1a None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:55.734582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3fb35-8025-4ebf-99c3-49e7705ecb1a None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:55.734856 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3fb35-8025-4ebf-99c3-49e7705ecb1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:55.741499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3fb35-8025-4ebf-99c3-49e7705ecb1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:55.743238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3fb35-8025-4ebf-99c3-49e7705ecb1a None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application [None req-28e3fb35-8025-4ebf-99c3-49e7705ecb1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 15:40:55.758241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application with Lock( May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:55.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:55.760250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 15:40:55.760250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 15:40:55.760250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 15:40:55.760250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 15:40:55.760250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 15:40:55.760250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 15:40:55.760250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 15:40:55.760250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: ERROR keystone.server.flask.application May 23 15:40:55.760250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 644/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Mon May 23 15:40:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:55.765926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6a4f1-cc95-4264-b113-c5ea2aa90f7a None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:55.821025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:55.875500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6a4f1-cc95-4264-b113-c5ea2aa90f7a None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:55.912598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab95be8-b787-4855-bc5f-04e373b2d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:55.913275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab95be8-b787-4855-bc5f-04e373b2d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:55.913557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab95be8-b787-4855-bc5f-04e373b2d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:55.913830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab95be8-b787-4855-bc5f-04e373b2d9d0 None admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:55.914630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab95be8-b787-4855-bc5f-04e373b2d9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:55.914940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab95be8-b787-4855-bc5f-04e373b2d9d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:55.916412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab95be8-b787-4855-bc5f-04e373b2d9d0 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:55.929745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 642/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Mon May 23 15:40:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:55.936821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3fb35-8025-4ebf-99c3-49e7705ecb1a None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:56.045668 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3fb35-8025-4ebf-99c3-49e7705ecb1a None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:56.082511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e164b05-983d-47e0-a7e4-0c19298f63a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:56.083099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e164b05-983d-47e0-a7e4-0c19298f63a6 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:56.083386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e164b05-983d-47e0-a7e4-0c19298f63a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:56.083666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e164b05-983d-47e0-a7e4-0c19298f63a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:56.084546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e164b05-983d-47e0-a7e4-0c19298f63a6 None admin] RBAC: Adding query filter params (role.id=04c11dea81274a69974ad5c639f2c60b, scope.project.id=7b8dd1fc12a644f79405a3ac265147b5, user.id=312ed2b555424c6a96b3d972ffea5ac0) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:56.084858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e164b05-983d-47e0-a7e4-0c19298f63a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:56.086677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e164b05-983d-47e0-a7e4-0c19298f63a6 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:56.093976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 645/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Mon May 23 15:40:55 2022] GET /identity/v3/role_assignments?user.id=312ed2b555424c6a96b3d972ffea5ac0&scope.project.id=7b8dd1fc12a644f79405a3ac265147b5&role.id=04c11dea81274a69974ad5c639f2c60b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:58.304964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694c418-7004-4dc3-ab58-04a9e9f25ec5 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:58.305378 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694c418-7004-4dc3-ab58-04a9e9f25ec5 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:58.305521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694c418-7004-4dc3-ab58-04a9e9f25ec5 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:58.306905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 643/1288] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:40:58.313172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd893fc-919c-48c3-9591-c019ae458d08 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:58.313726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd893fc-919c-48c3-9591-c019ae458d08 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:58.313726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd893fc-919c-48c3-9591-c019ae458d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:58.330021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-5fd893fc-919c-48c3-9591-c019ae458d08 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:40:58.331188 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5fd893fc-919c-48c3-9591-c019ae458d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:40:58.360724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:58.396179 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 646/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:40:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:40:58.418532 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694c418-7004-4dc3-ab58-04a9e9f25ec5 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:58.519579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694c418-7004-4dc3-ab58-04a9e9f25ec5 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:58.530929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:40:58.558077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab4d78-f6e6-41bc-9132-52f03f26b8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:58.558077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab4d78-f6e6-41bc-9132-52f03f26b8d6 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:58.558077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab4d78-f6e6-41bc-9132-52f03f26b8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:58.558077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab4d78-f6e6-41bc-9132-52f03f26b8d6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:58.558463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab4d78-f6e6-41bc-9132-52f03f26b8d6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:58.558463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab4d78-f6e6-41bc-9132-52f03f26b8d6 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-9eab4d78-f6e6-41bc-9132-52f03f26b8d6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:40:58.569340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:40:58.570524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:40:58.571685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 644/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 23 15:40:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:40:58.577222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd893fc-919c-48c3-9591-c019ae458d08 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:58.672960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd893fc-919c-48c3-9591-c019ae458d08 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:58.706269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b2f47-4272-4ec7-9841-df964f96cbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:58.706921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b2f47-4272-4ec7-9841-df964f96cbdd None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:58.707202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b2f47-4272-4ec7-9841-df964f96cbdd None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:58.707479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b2f47-4272-4ec7-9841-df964f96cbdd None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:58.708275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b2f47-4272-4ec7-9841-df964f96cbdd None admin] RBAC: Adding query filter params (name=placement) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:58.708648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b2f47-4272-4ec7-9841-df964f96cbdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:58.710030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b2f47-4272-4ec7-9841-df964f96cbdd None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:58.714801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 647/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Mon May 23 15:40:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:40:58.720566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab4d78-f6e6-41bc-9132-52f03f26b8d6 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:40:58.818535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab4d78-f6e6-41bc-9132-52f03f26b8d6 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:40:58.851296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621979a5-94b1-41c1-9f84-729dba312b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:40:58.851945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621979a5-94b1-41c1-9f84-729dba312b95 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:40:58.852223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621979a5-94b1-41c1-9f84-729dba312b95 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:40:58.852523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621979a5-94b1-41c1-9f84-729dba312b95 None admin] PATH_INFO: `/v3/services` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:40:58.853218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621979a5-94b1-41c1-9f84-729dba312b95 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:40:58.853522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621979a5-94b1-41c1-9f84-729dba312b95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:40:58.854855 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621979a5-94b1-41c1-9f84-729dba312b95 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:40:58.859694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 645/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Mon May 23 15:40:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:01.043615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe96edc-469d-4a18-adca-d804badd4453 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:01.044224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe96edc-469d-4a18-adca-d804badd4453 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:01.044224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe96edc-469d-4a18-adca-d804badd4453 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:01.050294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 648/1293] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:01.050933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38679-aa1e-4834-8b3b-6c1981106fec None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:01.051327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38679-aa1e-4834-8b3b-6c1981106fec None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:01.051486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38679-aa1e-4834-8b3b-6c1981106fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:01.068811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-cdd38679-aa1e-4834-8b3b-6c1981106fec None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:01.069976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-cdd38679-aa1e-4834-8b3b-6c1981106fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:01.099574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:01.135841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 646/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:41:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:41:01.158911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe96edc-469d-4a18-adca-d804badd4453 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:01.189592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:01.257479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe96edc-469d-4a18-adca-d804badd4453 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:01.292407 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1803c-933d-45c3-a284-c5f511270858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:01.293078 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1803c-933d-45c3-a284-c5f511270858 None admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:01.293363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1803c-933d-45c3-a284-c5f511270858 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:01.293688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1803c-933d-45c3-a284-c5f511270858 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:01.294370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1803c-933d-45c3-a284-c5f511270858 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:01.295732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1803c-933d-45c3-a284-c5f511270858 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:01.372705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 649/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Mon May 23 15:41:01 2022] POST /identity/v3/services => generated 246 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:41:03.544041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e230ee-154d-4425-927b-6a43d62aeb7e None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:03.544593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e230ee-154d-4425-927b-6a43d62aeb7e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:03.544593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e230ee-154d-4425-927b-6a43d62aeb7e None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:03.545941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 647/1296] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:03.551134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dfa5e-394d-4572-af47-325dbf3ff860 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:03.551571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dfa5e-394d-4572-af47-325dbf3ff860 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:03.551709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dfa5e-394d-4572-af47-325dbf3ff860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:03.568562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-467dfa5e-394d-4572-af47-325dbf3ff860 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:03.571483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-467dfa5e-394d-4572-af47-325dbf3ff860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:03.640968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 650/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:41:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:41:03.663931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e230ee-154d-4425-927b-6a43d62aeb7e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:03.772921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e230ee-154d-4425-927b-6a43d62aeb7e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:03.806803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0b839-9231-4e5d-89b0-39a46c198d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:03.807536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0b839-9231-4e5d-89b0-39a46c198d33 None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:03.807817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0b839-9231-4e5d-89b0-39a46c198d33 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:03.808091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0b839-9231-4e5d-89b0-39a46c198d33 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:03.808824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0b839-9231-4e5d-89b0-39a46c198d33 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:03.810696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0b839-9231-4e5d-89b0-39a46c198d33 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-34b0b839-9231-4e5d-89b0-39a46c198d33 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:41:03.822360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:41:03.827139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:41:03.828264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 648/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 23 15:41:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:41:03.829789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dfa5e-394d-4572-af47-325dbf3ff860 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:03.847481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:03.924658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dfa5e-394d-4572-af47-325dbf3ff860 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:03.958347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a4ea3-33af-4229-bd4e-9c861bf12a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:03.959010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a4ea3-33af-4229-bd4e-9c861bf12a51 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:03.959299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a4ea3-33af-4229-bd4e-9c861bf12a51 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:03.959583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a4ea3-33af-4229-bd4e-9c861bf12a51 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:03.960289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a4ea3-33af-4229-bd4e-9c861bf12a51 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:03.960632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a4ea3-33af-4229-bd4e-9c861bf12a51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:03.961996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a4ea3-33af-4229-bd4e-9c861bf12a51 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:03.967091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 651/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Mon May 23 15:41:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:03.973103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0b839-9231-4e5d-89b0-39a46c198d33 None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:04.004950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:04.071146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0b839-9231-4e5d-89b0-39a46c198d33 None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:04.104151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7445a8-05e5-41cc-83ef-0fa4da8a5bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:04.104834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7445a8-05e5-41cc-83ef-0fa4da8a5bec None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:04.105125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7445a8-05e5-41cc-83ef-0fa4da8a5bec None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:04.105404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7445a8-05e5-41cc-83ef-0fa4da8a5bec None admin] PATH_INFO: `/v3/endpoints` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:04.106789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7445a8-05e5-41cc-83ef-0fa4da8a5bec None admin] RBAC: Adding query filter params (interface=public, service_id=51dfcb1afddd4073aeecca3586af6c41, region_id=RegionOne) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:04.107105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7445a8-05e5-41cc-83ef-0fa4da8a5bec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:04.108531 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7445a8-05e5-41cc-83ef-0fa4da8a5bec None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:04.113454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 649/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Mon May 23 15:41:03 2022] GET /identity/v3/endpoints?service_id=51dfcb1afddd4073aeecca3586af6c41&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:04.119370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a4ea3-33af-4229-bd4e-9c861bf12a51 None admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:04.202494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:04.219624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a4ea3-33af-4229-bd4e-9c861bf12a51 None admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:04.252533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dfc5a-2bbe-40b3-bd8a-92f8a2bbdf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:04.253198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dfc5a-2bbe-40b3-bd8a-92f8a2bbdf35 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:04.253487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dfc5a-2bbe-40b3-bd8a-92f8a2bbdf35 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:04.253763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dfc5a-2bbe-40b3-bd8a-92f8a2bbdf35 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:04.254436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dfc5a-2bbe-40b3-bd8a-92f8a2bbdf35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:04.255816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dfc5a-2bbe-40b3-bd8a-92f8a2bbdf35 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:04.262996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 652/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Mon May 23 15:41:04 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:41:06.429353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2bf91-6f4d-469f-8877-6ae93696b729 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:06.429888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2bf91-6f4d-469f-8877-6ae93696b729 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:06.430071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2bf91-6f4d-469f-8877-6ae93696b729 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:06.431294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 650/1302] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:41:06.437105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a9f10-94d3-4348-b5fc-e9e9fad7b1fd None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:06.437522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a9f10-94d3-4348-b5fc-e9e9fad7b1fd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:06.437711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a9f10-94d3-4348-b5fc-e9e9fad7b1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:06.454892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-6a6a9f10-94d3-4348-b5fc-e9e9fad7b1fd None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:41:06.456068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6a6a9f10-94d3-4348-b5fc-e9e9fad7b1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:41:06.520419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 653/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:41:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 15:41:06.543053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2bf91-6f4d-469f-8877-6ae93696b729 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:06.635687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:06.642135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2bf91-6f4d-469f-8877-6ae93696b729 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:06.675070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a3425-ba07-4329-9e7b-21d25217db9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:06.675854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a3425-ba07-4329-9e7b-21d25217db9d None admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:06.676133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a3425-ba07-4329-9e7b-21d25217db9d None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:06.676432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a3425-ba07-4329-9e7b-21d25217db9d None admin] PATH_INFO: `/v3/services/placement` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:06.677080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a3425-ba07-4329-9e7b-21d25217db9d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:06.678427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a3425-ba07-4329-9e7b-21d25217db9d None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application [None req-759a3425-ba07-4329-9e7b-21d25217db9d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 15:41:06.691354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 15:41:06.692609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.get_or_create( May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application with Lock( May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self._enter() May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application return self.creator() May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application created_value = creator( May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: ERROR keystone.server.flask.application May 23 15:41:06.693753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 651/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Mon May 23 15:41:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 15:41:06.698552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a9f10-94d3-4348-b5fc-e9e9fad7b1fd None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:06.773531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:06.795716 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a9f10-94d3-4348-b5fc-e9e9fad7b1fd None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:06.829541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2d870-a08a-4195-a1c4-20e0ade93403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:06.830305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d870-a08a-4195-a1c4-20e0ade93403 None admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:06.830589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d870-a08a-4195-a1c4-20e0ade93403 None admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:06.830866 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2d870-a08a-4195-a1c4-20e0ade93403 None admin] PATH_INFO: `/v3/services` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:06.831559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2d870-a08a-4195-a1c4-20e0ade93403 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:41:06.831865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2d870-a08a-4195-a1c4-20e0ade93403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:06.833345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2d870-a08a-4195-a1c4-20e0ade93403 None admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:06.838340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 654/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Mon May 23 15:41:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:41:06.844206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a3425-ba07-4329-9e7b-21d25217db9d None admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:41:06.941708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a3425-ba07-4329-9e7b-21d25217db9d None admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:41:06.973974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0c48d-24f8-47cf-8a26-7d69d7838cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:41:06.974627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0c48d-24f8-47cf-8a26-7d69d7838cb4 None admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:41:06.974912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0c48d-24f8-47cf-8a26-7d69d7838cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:41:06.975188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0c48d-24f8-47cf-8a26-7d69d7838cb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:41:06.975949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0c48d-24f8-47cf-8a26-7d69d7838cb4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:41:06.977343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0c48d-24f8-47cf-8a26-7d69d7838cb4 None admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:41:07.063106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 652/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Mon May 23 15:41:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:41:16.846013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=30 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:41:17.058454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:11.034631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73601b83-2964-4109-bbcc-ffb63ed84693 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:11.035905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73601b83-2964-4109-bbcc-ffb63ed84693 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:11.036464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73601b83-2964-4109-bbcc-ffb63ed84693 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:11.038199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 655/1307] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:47:11.043475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a97e1f-9bf2-4fe0-9b5c-a7cdb39715ab None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:11.047354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a97e1f-9bf2-4fe0-9b5c-a7cdb39715ab None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:11.047354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a97e1f-9bf2-4fe0-9b5c-a7cdb39715ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:11.161945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b4a97e1f-9bf2-4fe0-9b5c-a7cdb39715ab None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:11.163837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b4a97e1f-9bf2-4fe0-9b5c-a7cdb39715ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:11.397258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 653/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:47:11.433360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a22513-5eaa-4498-8113-44cbc2b4e9c2 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:11.433730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a22513-5eaa-4498-8113-44cbc2b4e9c2 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:11.434205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a22513-5eaa-4498-8113-44cbc2b4e9c2 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:11.436116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 656/1309] 10.222.0.23 () {54 vars in 964 bytes} [Mon May 23 15:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:11.443118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5b418-a759-44a2-8245-ea1fa88b408a None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:11.443413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5b418-a759-44a2-8245-ea1fa88b408a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:11.443891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5b418-a759-44a2-8245-ea1fa88b408a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:11.540031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9df5b418-a759-44a2-8245-ea1fa88b408a None None] MFA Rules not processed for user `a39f5189d63d467dbd914f553c261c01`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:11.541494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9df5b418-a759-44a2-8245-ea1fa88b408a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:11.567077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:11.632306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 654/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Mon May 23 15:47:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:11.642037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a22513-5eaa-4498-8113-44cbc2b4e9c2 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:11.644120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-59a22513-5eaa-4498-8113-44cbc2b4e9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:11.758393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a22513-5eaa-4498-8113-44cbc2b4e9c2 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:11.835533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d447df-befd-4937-a894-4d3a068acf93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:11.836578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d447df-befd-4937-a894-4d3a068acf93 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:11.837185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d447df-befd-4937-a894-4d3a068acf93 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:11.837736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d447df-befd-4937-a894-4d3a068acf93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:11.839639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-82d447df-befd-4937-a894-4d3a068acf93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:11.896275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:11.956147 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d447df-befd-4937-a894-4d3a068acf93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:11.957975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d447df-befd-4937-a894-4d3a068acf93 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:11.992973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 657/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:14.511726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b26d-59a9-4c43-aff7-8e839167fbe1 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:14.512271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b26d-59a9-4c43-aff7-8e839167fbe1 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:14.512271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5b26d-59a9-4c43-aff7-8e839167fbe1 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:14.514304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 655/1312] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:47:14.520347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622b6f-3626-4f24-80b1-3466ed7fcafa None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:14.521120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622b6f-3626-4f24-80b1-3466ed7fcafa None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:14.521120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622b6f-3626-4f24-80b1-3466ed7fcafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:14.605589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-8b622b6f-3626-4f24-80b1-3466ed7fcafa None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:14.608023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8b622b6f-3626-4f24-80b1-3466ed7fcafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:14.715976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 658/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:14.753527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5b26d-59a9-4c43-aff7-8e839167fbe1 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:14.755053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c7f5b26d-59a9-4c43-aff7-8e839167fbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:14.859940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5b26d-59a9-4c43-aff7-8e839167fbe1 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:14.929732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:14.932344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf613ef-829d-4f00-8d7d-3983b7cfb3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:14.933011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf613ef-829d-4f00-8d7d-3983b7cfb3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:14.933289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf613ef-829d-4f00-8d7d-3983b7cfb3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:14.933560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf613ef-829d-4f00-8d7d-3983b7cfb3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:14.935170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-caf613ef-829d-4f00-8d7d-3983b7cfb3a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:15.042004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf613ef-829d-4f00-8d7d-3983b7cfb3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:15.043430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf613ef-829d-4f00-8d7d-3983b7cfb3a1 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:15.062201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 656/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:17.815725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e69042-f0d3-4e02-8517-001e0f616815 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:17.816281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e69042-f0d3-4e02-8517-001e0f616815 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:17.816281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e69042-f0d3-4e02-8517-001e0f616815 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:17.817720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 659/1315] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:17.822813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b3b45-3cf9-4550-afeb-c7d022b4444a None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:17.823193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b3b45-3cf9-4550-afeb-c7d022b4444a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:17.823375 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b3b45-3cf9-4550-afeb-c7d022b4444a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:17.886700 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-947b3b45-3cf9-4550-afeb-c7d022b4444a None None] MFA Rules not processed for user `7384fa3396274b4e8aa20b57838b11a6`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:17.887937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-947b3b45-3cf9-4550-afeb-c7d022b4444a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:17.938221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:17.980139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 657/1316] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:18.019818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e69042-f0d3-4e02-8517-001e0f616815 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:18.021397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c4e69042-f0d3-4e02-8517-001e0f616815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:18.030988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:18.120848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e69042-f0d3-4e02-8517-001e0f616815 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:18.191632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1390c81-34d3-4a22-ab2e-476d78640080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:18.192274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1390c81-34d3-4a22-ab2e-476d78640080 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:18.192579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1390c81-34d3-4a22-ab2e-476d78640080 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:18.192851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1390c81-34d3-4a22-ab2e-476d78640080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:18.194463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e1390c81-34d3-4a22-ab2e-476d78640080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:18.254684 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:18.301405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1390c81-34d3-4a22-ab2e-476d78640080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:18.302852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1390c81-34d3-4a22-ab2e-476d78640080 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:18.321390 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 660/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:21.416889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2b816-21aa-4d99-bae4-a17255934c9d None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:21.417404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2b816-21aa-4d99-bae4-a17255934c9d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:21.417519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2b816-21aa-4d99-bae4-a17255934c9d None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:21.418819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 658/1318] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:21.424158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8d27-c965-4077-bbc7-d5a6ef107983 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:21.424623 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8d27-c965-4077-bbc7-d5a6ef107983 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:21.424728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8d27-c965-4077-bbc7-d5a6ef107983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:21.488559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-67aa8d27-c965-4077-bbc7-d5a6ef107983 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:21.489760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-67aa8d27-c965-4077-bbc7-d5a6ef107983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:21.586107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 661/1319] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:21.623936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2b816-21aa-4d99-bae4-a17255934c9d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:21.626006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e3b2b816-21aa-4d99-bae4-a17255934c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:21.726219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2b816-21aa-4d99-bae4-a17255934c9d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:21.796468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1750db0-104d-4db5-a48c-a7b47dad6311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:21.797131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1750db0-104d-4db5-a48c-a7b47dad6311 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:21.797432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1750db0-104d-4db5-a48c-a7b47dad6311 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:21.797706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1750db0-104d-4db5-a48c-a7b47dad6311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:21.799310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d1750db0-104d-4db5-a48c-a7b47dad6311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:21.806371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:21.903166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1750db0-104d-4db5-a48c-a7b47dad6311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:21.904584 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1750db0-104d-4db5-a48c-a7b47dad6311 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:21.922635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 659/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:25.254708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8891689-4fec-4670-998e-425f513acdbc None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:25.255240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8891689-4fec-4670-998e-425f513acdbc None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:25.255240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8891689-4fec-4670-998e-425f513acdbc None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:25.256598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 662/1321] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:25.262128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fe06-dde6-4632-b128-940c7e4ff26b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:25.262552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fe06-dde6-4632-b128-940c7e4ff26b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:25.262807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fe06-dde6-4632-b128-940c7e4ff26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:25.328662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-71f1fe06-dde6-4632-b128-940c7e4ff26b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:25.329865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-71f1fe06-dde6-4632-b128-940c7e4ff26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:25.386383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:25.424283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 660/1322] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:25.460132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8891689-4fec-4670-998e-425f513acdbc None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:25.461725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c8891689-4fec-4670-998e-425f513acdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:25.482150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:25.568059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8891689-4fec-4670-998e-425f513acdbc None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:25.641981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76187ea6-d6d2-42c5-af51-de291454c6f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:25.642628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76187ea6-d6d2-42c5-af51-de291454c6f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:25.642908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76187ea6-d6d2-42c5-af51-de291454c6f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:25.643182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76187ea6-d6d2-42c5-af51-de291454c6f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:25.644888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-76187ea6-d6d2-42c5-af51-de291454c6f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:25.714726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:25.753046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76187ea6-d6d2-42c5-af51-de291454c6f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:25.754558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76187ea6-d6d2-42c5-af51-de291454c6f8 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:25.774106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 663/1323] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:28.016677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461a3a3-d2bf-4495-840f-9ecf89cd377c None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:28.017211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461a3a3-d2bf-4495-840f-9ecf89cd377c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:28.017211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461a3a3-d2bf-4495-840f-9ecf89cd377c None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:28.018616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 661/1324] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:28.023856 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703ef75-8467-47dc-abae-fe1d352d62dc None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:28.024244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703ef75-8467-47dc-abae-fe1d352d62dc None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:28.024453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703ef75-8467-47dc-abae-fe1d352d62dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:28.087368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2703ef75-8467-47dc-abae-fe1d352d62dc None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:28.088585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2703ef75-8467-47dc-abae-fe1d352d62dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:28.188284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 664/1325] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:28.239831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fdb3d-0b83-4259-b44b-30e00341d352 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:28.240119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fdb3d-0b83-4259-b44b-30e00341d352 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:28.240442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fdb3d-0b83-4259-b44b-30e00341d352 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:28.241820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 662/1326] 10.222.0.23 () {54 vars in 964 bytes} [Mon May 23 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:28.248130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a17733-bf24-4bc9-bef4-089e09a20171 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:28.248454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a17733-bf24-4bc9-bef4-089e09a20171 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:28.248733 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a17733-bf24-4bc9-bef4-089e09a20171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:28.279952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:28.327355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-33a17733-bf24-4bc9-bef4-089e09a20171 None None] MFA Rules not processed for user `a39f5189d63d467dbd914f553c261c01`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:28.328620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-33a17733-bf24-4bc9-bef4-089e09a20171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:28.404020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 665/1327] 10.222.0.23 () {58 vars in 1061 bytes} [Mon May 23 15:47:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:28.413840 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fdb3d-0b83-4259-b44b-30e00341d352 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:28.415344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9e2fdb3d-0b83-4259-b44b-30e00341d352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:28.517297 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fdb3d-0b83-4259-b44b-30e00341d352 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:28.589876 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe4daa-06f4-4038-927d-fb4760fec70e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:28.590455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe4daa-06f4-4038-927d-fb4760fec70e service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:28.590739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe4daa-06f4-4038-927d-fb4760fec70e service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:28.591020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe4daa-06f4-4038-927d-fb4760fec70e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:28.592772 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c7fe4daa-06f4-4038-927d-fb4760fec70e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:28.610030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:28.701426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe4daa-06f4-4038-927d-fb4760fec70e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:28.703442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe4daa-06f4-4038-927d-fb4760fec70e service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:28.722378 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 663/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:31.232808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02cc6-ea06-49f2-8378-e5c791b03bde None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:31.233362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02cc6-ea06-49f2-8378-e5c791b03bde None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:31.233515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02cc6-ea06-49f2-8378-e5c791b03bde None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:31.235348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 666/1329] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:31.240572 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e0f9c-2dc7-404d-a853-5a08133d5fe0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:31.240990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e0f9c-2dc7-404d-a853-5a08133d5fe0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:31.241244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e0f9c-2dc7-404d-a853-5a08133d5fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:31.306739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-574e0f9c-2dc7-404d-a853-5a08133d5fe0 None None] MFA Rules not processed for user `7384fa3396274b4e8aa20b57838b11a6`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:31.307922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-574e0f9c-2dc7-404d-a853-5a08133d5fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:31.370819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:31.401440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 664/1330] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:31.439337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02cc6-ea06-49f2-8378-e5c791b03bde None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:31.440959 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e9e02cc6-ea06-49f2-8378-e5c791b03bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:31.543213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02cc6-ea06-49f2-8378-e5c791b03bde None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:31.562637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:31.615210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913a339-6e7e-41fb-9481-c068d1e6eb1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:31.615862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913a339-6e7e-41fb-9481-c068d1e6eb1f service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:31.616145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913a339-6e7e-41fb-9481-c068d1e6eb1f service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:31.616452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913a339-6e7e-41fb-9481-c068d1e6eb1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:31.618086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8913a339-6e7e-41fb-9481-c068d1e6eb1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:31.724875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913a339-6e7e-41fb-9481-c068d1e6eb1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:31.726327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913a339-6e7e-41fb-9481-c068d1e6eb1f service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:31.744136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 667/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:34.719833 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c151e-043b-4b3f-8a07-6c4f073845b2 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:34.720352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c151e-043b-4b3f-8a07-6c4f073845b2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:34.720465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c151e-043b-4b3f-8a07-6c4f073845b2 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:34.721795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 665/1332] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:34.727077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c25f-868c-451c-862a-f18f8b1c7971 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:34.727471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c25f-868c-451c-862a-f18f8b1c7971 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:34.727656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9c25f-868c-451c-862a-f18f8b1c7971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:34.790990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:34.793558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-1eb9c25f-868c-451c-862a-f18f8b1c7971 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:34.794784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1eb9c25f-868c-451c-862a-f18f8b1c7971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:34.891762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 668/1333] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:34.926448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c151e-043b-4b3f-8a07-6c4f073845b2 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:34.927983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-231c151e-043b-4b3f-8a07-6c4f073845b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:35.037860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c151e-043b-4b3f-8a07-6c4f073845b2 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:35.113655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1a9e8-9d6d-404b-9622-abb2c4091976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:35.114321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1a9e8-9d6d-404b-9622-abb2c4091976 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:35.114604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1a9e8-9d6d-404b-9622-abb2c4091976 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:35.114879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1a9e8-9d6d-404b-9622-abb2c4091976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:35.116563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e9c1a9e8-9d6d-404b-9622-abb2c4091976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:35.146356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:35.232847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1a9e8-9d6d-404b-9622-abb2c4091976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:35.234981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1a9e8-9d6d-404b-9622-abb2c4091976 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:35.254246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 666/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:37.500206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8ada3-75ca-4d1b-883d-d5e3b9202db5 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:37.500808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8ada3-75ca-4d1b-883d-d5e3b9202db5 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:37.500808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8ada3-75ca-4d1b-883d-d5e3b9202db5 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:37.502174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 669/1335] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:37.507569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f56a61-a799-488d-a91a-da5036edda89 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:37.507966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f56a61-a799-488d-a91a-da5036edda89 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:37.508131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f56a61-a799-488d-a91a-da5036edda89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:37.571750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e9f56a61-a799-488d-a91a-da5036edda89 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:37.572933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e9f56a61-a799-488d-a91a-da5036edda89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:37.647162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:37.670338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 667/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:37.709750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8ada3-75ca-4d1b-883d-d5e3b9202db5 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:37.711290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f0c8ada3-75ca-4d1b-883d-d5e3b9202db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:37.814164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8ada3-75ca-4d1b-883d-d5e3b9202db5 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:37.841714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:37.886598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3888e73-2591-43c6-8265-d1e97f2a936b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:37.887243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3888e73-2591-43c6-8265-d1e97f2a936b service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:37.887524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3888e73-2591-43c6-8265-d1e97f2a936b service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:37.887802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3888e73-2591-43c6-8265-d1e97f2a936b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:37.889475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b3888e73-2591-43c6-8265-d1e97f2a936b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:37.995255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3888e73-2591-43c6-8265-d1e97f2a936b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:37.996796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3888e73-2591-43c6-8265-d1e97f2a936b service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:38.015613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 670/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:40.985503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb3c4-9c45-4a58-85a0-2022e350f7f6 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:40.986114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb3c4-9c45-4a58-85a0-2022e350f7f6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:40.986114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecb3c4-9c45-4a58-85a0-2022e350f7f6 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:40.987444 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 668/1338] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:40.992716 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680dcc-f948-4d14-99c8-ead5987a2950 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:40.993155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680dcc-f948-4d14-99c8-ead5987a2950 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:40.993386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680dcc-f948-4d14-99c8-ead5987a2950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:41.057822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-60680dcc-f948-4d14-99c8-ead5987a2950 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:41.059080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-60680dcc-f948-4d14-99c8-ead5987a2950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:41.067641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:41.154588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 671/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:41.190194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecb3c4-9c45-4a58-85a0-2022e350f7f6 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:41.193228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-15ecb3c4-9c45-4a58-85a0-2022e350f7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:41.303260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecb3c4-9c45-4a58-85a0-2022e350f7f6 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:41.315775 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf06f5b-cfc5-4ec0-92e5-9aefc69422bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:41.316447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf06f5b-cfc5-4ec0-92e5-9aefc69422bf service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:41.316731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf06f5b-cfc5-4ec0-92e5-9aefc69422bf service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:41.317003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf06f5b-cfc5-4ec0-92e5-9aefc69422bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:41.318586 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-bdf06f5b-cfc5-4ec0-92e5-9aefc69422bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:41.419984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:47:41.422933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf06f5b-cfc5-4ec0-92e5-9aefc69422bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:41.424606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf06f5b-cfc5-4ec0-92e5-9aefc69422bf service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:41.444744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 669/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:46.133909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063a913-dd29-4f9e-bbe3-12e4998d19bd None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:46.134412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063a913-dd29-4f9e-bbe3-12e4998d19bd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:46.134507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063a913-dd29-4f9e-bbe3-12e4998d19bd None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:46.135944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 672/1341] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:46.142932 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee01b-f636-4335-9374-798914d75860 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:46.143384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee01b-f636-4335-9374-798914d75860 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:46.143647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee01b-f636-4335-9374-798914d75860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:46.190996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-11bee01b-f636-4335-9374-798914d75860 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:46.192180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-11bee01b-f636-4335-9374-798914d75860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:46.276413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 670/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:46.315697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063a913-dd29-4f9e-bbe3-12e4998d19bd None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:46.321932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0063a913-dd29-4f9e-bbe3-12e4998d19bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:46.342892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063a913-dd29-4f9e-bbe3-12e4998d19bd None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:46.346483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7aa87-baa4-41ba-a5f2-458d86ba10e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:46.347141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7aa87-baa4-41ba-a5f2-458d86ba10e5 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:46.347425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7aa87-baa4-41ba-a5f2-458d86ba10e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:46.347702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7aa87-baa4-41ba-a5f2-458d86ba10e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:46.357618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7aa87-baa4-41ba-a5f2-458d86ba10e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:46.360194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7aa87-baa4-41ba-a5f2-458d86ba10e5 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:46.369884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 673/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:49.711735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b04d-0899-4433-8bb6-64fb769591a8 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:49.712321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b04d-0899-4433-8bb6-64fb769591a8 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:49.712321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b04d-0899-4433-8bb6-64fb769591a8 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:49.713749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 671/1344] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:49.719710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd9c47-35ce-4fd6-a78d-566591cd6afd None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:49.720210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd9c47-35ce-4fd6-a78d-566591cd6afd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:49.720326 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd9c47-35ce-4fd6-a78d-566591cd6afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:49.750737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ebdd9c47-35ce-4fd6-a78d-566591cd6afd None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:49.751931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ebdd9c47-35ce-4fd6-a78d-566591cd6afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:49.762434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 674/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:49.798513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81b04d-0899-4433-8bb6-64fb769591a8 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:49.809287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81b04d-0899-4433-8bb6-64fb769591a8 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:49.813479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c901e-38cc-4d86-a0f2-872e14922be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:49.814113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c901e-38cc-4d86-a0f2-872e14922be4 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:49.814397 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c901e-38cc-4d86-a0f2-872e14922be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:49.814673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c901e-38cc-4d86-a0f2-872e14922be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:49.824130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c901e-38cc-4d86-a0f2-872e14922be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:49.825590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c901e-38cc-4d86-a0f2-872e14922be4 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:49.835151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 672/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:54.360319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f955854-eccf-464e-84ed-9158ca77052d None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:54.360979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f955854-eccf-464e-84ed-9158ca77052d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:54.360979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f955854-eccf-464e-84ed-9158ca77052d None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:54.362247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 675/1347] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:54.367422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d824e25-8369-4462-88fa-1c62a5366b90 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:54.367814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d824e25-8369-4462-88fa-1c62a5366b90 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:54.367982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d824e25-8369-4462-88fa-1c62a5366b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:54.386451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1d824e25-8369-4462-88fa-1c62a5366b90 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:47:54.387631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1d824e25-8369-4462-88fa-1c62a5366b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:54.395956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 673/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:47:54.432659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f955854-eccf-464e-84ed-9158ca77052d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:47:54.434386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3f955854-eccf-464e-84ed-9158ca77052d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:47:54.448404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f955854-eccf-464e-84ed-9158ca77052d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:47:54.452163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1a42f-b7eb-40f8-a5bb-78c6602df491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:47:54.453484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1a42f-b7eb-40f8-a5bb-78c6602df491 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:54.453768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1a42f-b7eb-40f8-a5bb-78c6602df491 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:54.454044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1a42f-b7eb-40f8-a5bb-78c6602df491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:54.463263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1a42f-b7eb-40f8-a5bb-78c6602df491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:47:54.464869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1a42f-b7eb-40f8-a5bb-78c6602df491 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:47:54.474760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 676/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:47:59.779472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab964e6-f8f9-42a0-b7d0-2daf9b1c63b0 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:59.779968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab964e6-f8f9-42a0-b7d0-2daf9b1c63b0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:59.780185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab964e6-f8f9-42a0-b7d0-2daf9b1c63b0 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:47:59.782141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 674/1350] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 15:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:47:59.787328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec780f1-5343-4a73-a682-e7a455b1f8b4 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:47:59.788196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec780f1-5343-4a73-a682-e7a455b1f8b4 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:47:59.788893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec780f1-5343-4a73-a682-e7a455b1f8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:00.004820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-5ec780f1-5343-4a73-a682-e7a455b1f8b4 None None] MFA Rules not processed for user `312ed2b555424c6a96b3d972ffea5ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:00.006014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5ec780f1-5343-4a73-a682-e7a455b1f8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:00.046913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:00.083549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 677/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 15:47:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:04.242548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d82ac-ea97-4f05-87d5-f6177d18d510 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:04.243111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d82ac-ea97-4f05-87d5-f6177d18d510 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:04.243111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d82ac-ea97-4f05-87d5-f6177d18d510 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:04.244776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 675/1352] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 15:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:04.249923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb5215-69c6-4603-98f7-4aea2a75b657 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:04.250418 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb5215-69c6-4603-98f7-4aea2a75b657 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:04.250499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb5215-69c6-4603-98f7-4aea2a75b657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:04.269255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-58bb5215-69c6-4603-98f7-4aea2a75b657 None None] MFA Rules not processed for user `312ed2b555424c6a96b3d972ffea5ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:04.270466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-58bb5215-69c6-4603-98f7-4aea2a75b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:04.283995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 678/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:04.380763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=60 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:05.778137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2800d5-0793-4af0-a1f8-a10d9592c6af None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:05.778784 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2800d5-0793-4af0-a1f8-a10d9592c6af None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:05.778784 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2800d5-0793-4af0-a1f8-a10d9592c6af None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:05.780259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 676/1354] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 15:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:05.785471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4f68a-28e0-48bd-92d2-106883c91f93 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:05.785902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4f68a-28e0-48bd-92d2-106883c91f93 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:05.786035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4f68a-28e0-48bd-92d2-106883c91f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:05.805649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-64c4f68a-28e0-48bd-92d2-106883c91f93 None None] MFA Rules not processed for user `312ed2b555424c6a96b3d972ffea5ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:05.807595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-64c4f68a-28e0-48bd-92d2-106883c91f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:05.815784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 679/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 15:48:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:10.184421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930672-d2c6-4e14-9a76-9e276ad7b435 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:10.184421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930672-d2c6-4e14-9a76-9e276ad7b435 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:10.184421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930672-d2c6-4e14-9a76-9e276ad7b435 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:10.184421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 677/1356] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:10.186395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe63443-f178-4ea9-9f54-28c0b766b7a1 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:10.186395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe63443-f178-4ea9-9f54-28c0b766b7a1 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:10.186395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe63443-f178-4ea9-9f54-28c0b766b7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:10.186395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-afe63443-f178-4ea9-9f54-28c0b766b7a1 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:10.186395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-afe63443-f178-4ea9-9f54-28c0b766b7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:10.186395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 680/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:10.190057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc3f1f-fc06-4aac-830d-b24abccca636 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:10.190561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc3f1f-fc06-4aac-830d-b24abccca636 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:10.191030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc3f1f-fc06-4aac-830d-b24abccca636 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:10.192912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 678/1358] 10.222.0.23 () {54 vars in 961 bytes} [Mon May 23 15:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:10.202774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708d509-7fbb-4269-af64-2ec78f360082 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:10.203285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708d509-7fbb-4269-af64-2ec78f360082 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:10.203902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708d509-7fbb-4269-af64-2ec78f360082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:10.258369 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-3708d509-7fbb-4269-af64-2ec78f360082 None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:10.259533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3708d509-7fbb-4269-af64-2ec78f360082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:10.364203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 681/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Mon May 23 15:48:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:48:10.365296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc3f1f-fc06-4aac-830d-b24abccca636 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:10.380673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc3f1f-fc06-4aac-830d-b24abccca636 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:10.386583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84460c0-d7ef-4e6b-88bd-60b44f70f4e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:10.387226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84460c0-d7ef-4e6b-88bd-60b44f70f4e5 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:10.387510 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84460c0-d7ef-4e6b-88bd-60b44f70f4e5 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:10.387807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84460c0-d7ef-4e6b-88bd-60b44f70f4e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:10.397943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84460c0-d7ef-4e6b-88bd-60b44f70f4e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:10.399533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84460c0-d7ef-4e6b-88bd-60b44f70f4e5 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:10.410224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 679/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:48:12.851802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce7525-e4a0-43fd-9f23-5efe4d47f8bf None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:12.852423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce7525-e4a0-43fd-9f23-5efe4d47f8bf None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:12.852740 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce7525-e4a0-43fd-9f23-5efe4d47f8bf None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:12.854405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 682/1361] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:12.860343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb71b10-2470-4d65-925d-0e3349433d0b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:12.860837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb71b10-2470-4d65-925d-0e3349433d0b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:12.861131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb71b10-2470-4d65-925d-0e3349433d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:12.881841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ebb71b10-2470-4d65-925d-0e3349433d0b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:12.883294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ebb71b10-2470-4d65-925d-0e3349433d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:12.891326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 680/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:12.935507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce7525-e4a0-43fd-9f23-5efe4d47f8bf None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:12.951772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce7525-e4a0-43fd-9f23-5efe4d47f8bf None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:12.958532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0f7e2-4e42-4d09-a403-dcfda6e916f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:12.959420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0f7e2-4e42-4d09-a403-dcfda6e916f0 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:12.960158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0f7e2-4e42-4d09-a403-dcfda6e916f0 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:12.960684 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0f7e2-4e42-4d09-a403-dcfda6e916f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:12.971178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0f7e2-4e42-4d09-a403-dcfda6e916f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:12.973264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0f7e2-4e42-4d09-a403-dcfda6e916f0 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:12.985387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 683/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:15.857574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36884401-18d3-4fc7-847a-8ab957db1206 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.858040 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36884401-18d3-4fc7-847a-8ab957db1206 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.858135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36884401-18d3-4fc7-847a-8ab957db1206 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.860277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 681/1364] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:15.865736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ad136-35d3-4d57-86bd-d981ec48167d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.866168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ad136-35d3-4d57-86bd-d981ec48167d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.866475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ad136-35d3-4d57-86bd-d981ec48167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.884554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d85ad136-35d3-4d57-86bd-d981ec48167d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:15.885781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d85ad136-35d3-4d57-86bd-d981ec48167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:15.899874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 684/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:15.947112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7289-4e81-46d6-b26f-54b9e44fb0c4 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.947402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7289-4e81-46d6-b26f-54b9e44fb0c4 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.947687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7289-4e81-46d6-b26f-54b9e44fb0c4 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.949102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 682/1366] 10.222.0.23 () {54 vars in 961 bytes} [Mon May 23 15:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:15.955318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00550b6f-2f84-43e7-8f41-1b3a201800fb None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:15.963542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00550b6f-2f84-43e7-8f41-1b3a201800fb None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:15.963542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00550b6f-2f84-43e7-8f41-1b3a201800fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:15.979486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-00550b6f-2f84-43e7-8f41-1b3a201800fb None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:15.981660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-00550b6f-2f84-43e7-8f41-1b3a201800fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:15.996012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 685/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Mon May 23 15:48:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:16.007352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb7289-4e81-46d6-b26f-54b9e44fb0c4 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:16.028901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb7289-4e81-46d6-b26f-54b9e44fb0c4 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:16.035405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd332dd-6674-405e-bfd0-6d21c95cbfb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:16.036077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd332dd-6674-405e-bfd0-6d21c95cbfb6 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:16.036353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd332dd-6674-405e-bfd0-6d21c95cbfb6 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:16.036682 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd332dd-6674-405e-bfd0-6d21c95cbfb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:16.045772 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd332dd-6674-405e-bfd0-6d21c95cbfb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:16.047939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd332dd-6674-405e-bfd0-6d21c95cbfb6 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:16.058201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 683/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:18.326016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bebf3-970b-4c88-9d9a-ad987f2853fd None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:18.326552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bebf3-970b-4c88-9d9a-ad987f2853fd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:18.326682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bebf3-970b-4c88-9d9a-ad987f2853fd None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:18.328545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 686/1369] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:18.334238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230e96-af9f-4084-83b2-07a420435d43 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:18.334743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230e96-af9f-4084-83b2-07a420435d43 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:18.334839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230e96-af9f-4084-83b2-07a420435d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:18.354076 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-18230e96-af9f-4084-83b2-07a420435d43 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:18.355301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-18230e96-af9f-4084-83b2-07a420435d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:18.363474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 684/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:18.636091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ad5fb-693f-470f-baf8-ca230a5911c8 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:18.636542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ad5fb-693f-470f-baf8-ca230a5911c8 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:18.636861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ad5fb-693f-470f-baf8-ca230a5911c8 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:18.638608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 687/1371] 10.222.0.23 () {54 vars in 963 bytes} [Mon May 23 15:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:18.644178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f3e4f-c793-4ac8-b2d3-6ca271e84d35 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:18.644529 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f3e4f-c793-4ac8-b2d3-6ca271e84d35 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:18.644821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f3e4f-c793-4ac8-b2d3-6ca271e84d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:18.698166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-658f3e4f-c793-4ac8-b2d3-6ca271e84d35 None None] MFA Rules not processed for user `0d30c8d1f883400ba397ea09d42a0aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:18.699353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-658f3e4f-c793-4ac8-b2d3-6ca271e84d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:18.767344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 685/1372] 10.222.0.23 () {58 vars in 1060 bytes} [Mon May 23 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:18.771080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091346c2-877f-442f-8c02-e7dc6dd0b1d2 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:18.771375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091346c2-877f-442f-8c02-e7dc6dd0b1d2 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:18.771657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091346c2-877f-442f-8c02-e7dc6dd0b1d2 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:18.773181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 688/1373] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:18.775487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f3e4f-c793-4ac8-b2d3-6ca271e84d35 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:18.778091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca2971-f712-47ce-bdc8-1afab692c3b4 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:18.778383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca2971-f712-47ce-bdc8-1afab692c3b4 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:18.778662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca2971-f712-47ce-bdc8-1afab692c3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:18.784526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f3e4f-c793-4ac8-b2d3-6ca271e84d35 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:18.788408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6be06-e239-4582-bb77-58f564552407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:18.789073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6be06-e239-4582-bb77-58f564552407 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:18.789367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6be06-e239-4582-bb77-58f564552407 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:18.789654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6be06-e239-4582-bb77-58f564552407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:18.798352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-8cca2971-f712-47ce-bdc8-1afab692c3b4 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:18.799555 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8cca2971-f712-47ce-bdc8-1afab692c3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:18.800063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6be06-e239-4582-bb77-58f564552407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:18.801652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6be06-e239-4582-bb77-58f564552407 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:18.808041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 689/1374] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:48:18.812986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 686/1375] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:18.851863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca2971-f712-47ce-bdc8-1afab692c3b4 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:18.864799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca2971-f712-47ce-bdc8-1afab692c3b4 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:18.870267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5940c25-ae8d-40d3-9880-0c8c6e927f1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:18.870917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5940c25-ae8d-40d3-9880-0c8c6e927f1f service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:18.871225 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5940c25-ae8d-40d3-9880-0c8c6e927f1f service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:18.871480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5940c25-ae8d-40d3-9880-0c8c6e927f1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:18.880321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5940c25-ae8d-40d3-9880-0c8c6e927f1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:18.881883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5940c25-ae8d-40d3-9880-0c8c6e927f1f service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:18.892044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 690/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:21.331995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be49fc1-7ea4-4029-9c6b-bc1df823897d None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:21.332590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be49fc1-7ea4-4029-9c6b-bc1df823897d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:21.332590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be49fc1-7ea4-4029-9c6b-bc1df823897d None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:21.333970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 687/1377] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:21.339798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd3f9c-b8e1-4ef8-b242-c928c3aea4fe None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:21.340256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd3f9c-b8e1-4ef8-b242-c928c3aea4fe None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:21.340390 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd3f9c-b8e1-4ef8-b242-c928c3aea4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:21.359824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-11fd3f9c-b8e1-4ef8-b242-c928c3aea4fe None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:21.369242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-11fd3f9c-b8e1-4ef8-b242-c928c3aea4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:21.369242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 691/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:21.416160 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be49fc1-7ea4-4029-9c6b-bc1df823897d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:21.436056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be49fc1-7ea4-4029-9c6b-bc1df823897d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:21.442451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da434a-33af-4ac9-a895-785371fa729e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:21.443272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da434a-33af-4ac9-a895-785371fa729e service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:21.443272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da434a-33af-4ac9-a895-785371fa729e service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:21.443485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da434a-33af-4ac9-a895-785371fa729e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:21.452382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da434a-33af-4ac9-a895-785371fa729e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:21.454514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da434a-33af-4ac9-a895-785371fa729e service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:21.465399 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 688/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:24.169895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3aebf-5251-413c-9087-76aa43a54352 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:24.170494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3aebf-5251-413c-9087-76aa43a54352 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:24.170494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3aebf-5251-413c-9087-76aa43a54352 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:24.172725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 692/1380] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:24.178287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71f3b1-ed5c-422c-9552-f058fb7df940 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:24.178702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71f3b1-ed5c-422c-9552-f058fb7df940 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:24.178837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71f3b1-ed5c-422c-9552-f058fb7df940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:24.197941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ec71f3b1-ed5c-422c-9552-f058fb7df940 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:24.199119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ec71f3b1-ed5c-422c-9552-f058fb7df940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:24.207147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 689/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:24.258398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3aebf-5251-413c-9087-76aa43a54352 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:24.273672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3aebf-5251-413c-9087-76aa43a54352 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:24.279499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e1703-b36c-4fc8-b596-ef31045caf09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:24.280335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e1703-b36c-4fc8-b596-ef31045caf09 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:24.281308 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e1703-b36c-4fc8-b596-ef31045caf09 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:24.281819 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e1703-b36c-4fc8-b596-ef31045caf09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:24.290305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:24.293579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e1703-b36c-4fc8-b596-ef31045caf09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:24.295646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e1703-b36c-4fc8-b596-ef31045caf09 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:24.307675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 693/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:26.840827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af0af-0411-421c-8f17-8da618513cd5 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:26.841506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af0af-0411-421c-8f17-8da618513cd5 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:26.841811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af0af-0411-421c-8f17-8da618513cd5 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:26.849265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 690/1383] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:26.849591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e864ed-8cb5-45cd-b8ca-19c22fa2c7c2 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:26.849591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e864ed-8cb5-45cd-b8ca-19c22fa2c7c2 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:26.850328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e864ed-8cb5-45cd-b8ca-19c22fa2c7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:26.867917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d2e864ed-8cb5-45cd-b8ca-19c22fa2c7c2 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:26.869770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d2e864ed-8cb5-45cd-b8ca-19c22fa2c7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:26.879978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 694/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:26.921561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2af0af-0411-421c-8f17-8da618513cd5 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:26.936181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2af0af-0411-421c-8f17-8da618513cd5 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:26.942006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826a2c7-0dba-4811-9be9-97c7663965de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:26.942710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826a2c7-0dba-4811-9be9-97c7663965de service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:26.942710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826a2c7-0dba-4811-9be9-97c7663965de service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:26.943111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826a2c7-0dba-4811-9be9-97c7663965de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:26.955021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826a2c7-0dba-4811-9be9-97c7663965de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:26.956694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826a2c7-0dba-4811-9be9-97c7663965de service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:26.970945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 691/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:29.260060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815501f-92c8-460d-a713-dbc7695895bc None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:29.260759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815501f-92c8-460d-a713-dbc7695895bc None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:29.260759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815501f-92c8-460d-a713-dbc7695895bc None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:29.262038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 695/1386] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:29.267658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3b844-6276-4058-bad4-be701f3913d7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:29.268107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3b844-6276-4058-bad4-be701f3913d7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:29.268222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3b844-6276-4058-bad4-be701f3913d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:29.288004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8bd3b844-6276-4058-bad4-be701f3913d7 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:29.289566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8bd3b844-6276-4058-bad4-be701f3913d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:29.297824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 692/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:29.341406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815501f-92c8-460d-a713-dbc7695895bc None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:29.356098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815501f-92c8-460d-a713-dbc7695895bc None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:29.362151 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57539020-00c1-4d5a-afea-ce01ae93fb23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:29.362798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57539020-00c1-4d5a-afea-ce01ae93fb23 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:29.363086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57539020-00c1-4d5a-afea-ce01ae93fb23 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:29.363364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57539020-00c1-4d5a-afea-ce01ae93fb23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:29.371719 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57539020-00c1-4d5a-afea-ce01ae93fb23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:29.373457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57539020-00c1-4d5a-afea-ce01ae93fb23 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:29.383232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 696/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:31.867969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecf1b7-bf94-4e35-a31b-3635265c0c6c None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:31.868713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecf1b7-bf94-4e35-a31b-3635265c0c6c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:31.868713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecf1b7-bf94-4e35-a31b-3635265c0c6c None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:31.869920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 693/1389] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:48:31.875735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d7007-0023-47ed-bb90-9f0a08b0ba50 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:31.876193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d7007-0023-47ed-bb90-9f0a08b0ba50 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:31.876351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d7007-0023-47ed-bb90-9f0a08b0ba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:31.895165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-df9d7007-0023-47ed-bb90-9f0a08b0ba50 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:31.896374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-df9d7007-0023-47ed-bb90-9f0a08b0ba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:31.904105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 697/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:48:31.948923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecf1b7-bf94-4e35-a31b-3635265c0c6c None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:31.963933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecf1b7-bf94-4e35-a31b-3635265c0c6c None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:31.969606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917b0f1-becb-47d3-9669-29e0bc66ef39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:31.970270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b0f1-becb-47d3-9669-29e0bc66ef39 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:31.970566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b0f1-becb-47d3-9669-29e0bc66ef39 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:31.970851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917b0f1-becb-47d3-9669-29e0bc66ef39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:31.979576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917b0f1-becb-47d3-9669-29e0bc66ef39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:31.981259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917b0f1-becb-47d3-9669-29e0bc66ef39 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:31.991913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 694/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:34.459406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a65311-4718-48e4-a9d5-991379caa192 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:34.460094 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a65311-4718-48e4-a9d5-991379caa192 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:34.460094 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a65311-4718-48e4-a9d5-991379caa192 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:34.461348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 698/1392] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:34.466665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f2108-63fa-405e-ad6a-2e508703ce6d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:34.467140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f2108-63fa-405e-ad6a-2e508703ce6d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:34.467230 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f2108-63fa-405e-ad6a-2e508703ce6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:34.473916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=87 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:34.486211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1e3f2108-63fa-405e-ad6a-2e508703ce6d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:34.487458 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1e3f2108-63fa-405e-ad6a-2e508703ce6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:34.495666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 695/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:34.540477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a65311-4718-48e4-a9d5-991379caa192 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:34.554523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a65311-4718-48e4-a9d5-991379caa192 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:34.559953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639dae1-f310-4587-84ad-8a5145f6bbfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:34.560671 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639dae1-f310-4587-84ad-8a5145f6bbfa service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:34.560957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639dae1-f310-4587-84ad-8a5145f6bbfa service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:34.561231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639dae1-f310-4587-84ad-8a5145f6bbfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:34.570246 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639dae1-f310-4587-84ad-8a5145f6bbfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:34.571825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639dae1-f310-4587-84ad-8a5145f6bbfa service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:34.581964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 699/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:34.838852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fcb86-6f8c-444a-9ecc-05065c523612 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:34.839147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fcb86-6f8c-444a-9ecc-05065c523612 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:34.839430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fcb86-6f8c-444a-9ecc-05065c523612 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:34.841031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 696/1395] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:34.847404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fe45b-df95-47b2-9f87-b05b2e49271c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:34.847724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fe45b-df95-47b2-9f87-b05b2e49271c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:34.848010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fe45b-df95-47b2-9f87-b05b2e49271c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:34.867315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e19fe45b-df95-47b2-9f87-b05b2e49271c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:34.868601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e19fe45b-df95-47b2-9f87-b05b2e49271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:34.877278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 700/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:34.935901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babbfb1-2efa-4e9b-a54d-3b82f4c9f9cd None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:34.936187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babbfb1-2efa-4e9b-a54d-3b82f4c9f9cd None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:34.936522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babbfb1-2efa-4e9b-a54d-3b82f4c9f9cd None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:34.938543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 697/1397] 10.222.0.23 () {54 vars in 962 bytes} [Mon May 23 15:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:34.944420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ffde1-1adb-4a1d-a9b7-9c97a3c8b16f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:34.945073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ffde1-1adb-4a1d-a9b7-9c97a3c8b16f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:34.945428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ffde1-1adb-4a1d-a9b7-9c97a3c8b16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:35.000900 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ca7ffde1-1adb-4a1d-a9b7-9c97a3c8b16f None None] MFA Rules not processed for user `142e6e44d6544f6a891ed3f913de74ae`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:35.002116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ca7ffde1-1adb-4a1d-a9b7-9c97a3c8b16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:35.064350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 701/1398] 10.222.0.23 () {58 vars in 1059 bytes} [Mon May 23 15:48:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:35.072275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babbfb1-2efa-4e9b-a54d-3b82f4c9f9cd None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:35.083495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babbfb1-2efa-4e9b-a54d-3b82f4c9f9cd None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:35.086917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e31087-5985-4e86-873d-3a9049cb2622 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '142e6e44d6544f6a891ed3f913de74ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:35.087561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e31087-5985-4e86-873d-3a9049cb2622 service glance] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:35.087844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e31087-5985-4e86-873d-3a9049cb2622 service glance] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:35.088118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e31087-5985-4e86-873d-3a9049cb2622 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:35.097912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e31087-5985-4e86-873d-3a9049cb2622 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:35.099430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e31087-5985-4e86-873d-3a9049cb2622 service glance] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:35.109052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 698/1399] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:35.865175 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6650ec4-0970-4291-b07d-ff429385025f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:35.869021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6650ec4-0970-4291-b07d-ff429385025f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:35.869021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6650ec4-0970-4291-b07d-ff429385025f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:35.924202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d6650ec4-0970-4291-b07d-ff429385025f None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:35.925454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d6650ec4-0970-4291-b07d-ff429385025f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:35.985302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:35.992415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 702/1400] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 15:48:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:36.031711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f2b8-6d3e-4924-818d-a2958e7c6cd0 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:36.032241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f2b8-6d3e-4924-818d-a2958e7c6cd0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:36.032407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f2b8-6d3e-4924-818d-a2958e7c6cd0 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:36.034059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 699/1401] 10.222.0.23 () {54 vars in 942 bytes} [Mon May 23 15:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:36.040724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee35aee-6aed-43d4-b8da-f3e5fe44db3b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:36.041079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee35aee-6aed-43d4-b8da-f3e5fe44db3b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:36.046329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee35aee-6aed-43d4-b8da-f3e5fe44db3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:36.090822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-dee35aee-6aed-43d4-b8da-f3e5fe44db3b None None] MFA Rules not processed for user `e57de80064b74ad59d33abb83a5d17f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:36.092106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-dee35aee-6aed-43d4-b8da-f3e5fe44db3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:36.164081 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 703/1402] 10.222.0.23 () {58 vars in 1039 bytes} [Mon May 23 15:48:36 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:36.173955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f2b8-6d3e-4924-818d-a2958e7c6cd0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:36.187301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f2b8-6d3e-4924-818d-a2958e7c6cd0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:36.192323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95324f6c-57c7-4bec-aec1-3a81105dc1a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:36.193079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95324f6c-57c7-4bec-aec1-3a81105dc1a7 service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:36.193433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95324f6c-57c7-4bec-aec1-3a81105dc1a7 service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:36.193781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95324f6c-57c7-4bec-aec1-3a81105dc1a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:36.201739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95324f6c-57c7-4bec-aec1-3a81105dc1a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:36.203386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95324f6c-57c7-4bec-aec1-3a81105dc1a7 service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:36.211456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 700/1403] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 15:48:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:48:37.021800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6a14b-83b6-4d2b-bdbc-5794b8f74aa4 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:37.022394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6a14b-83b6-4d2b-bdbc-5794b8f74aa4 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:37.022394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6a14b-83b6-4d2b-bdbc-5794b8f74aa4 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:37.023909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 704/1404] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:37.029490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7edf2-ab68-4c3f-8813-40a705ece5c6 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:37.030018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7edf2-ab68-4c3f-8813-40a705ece5c6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:37.030516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7edf2-ab68-4c3f-8813-40a705ece5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:37.053173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-33b7edf2-ab68-4c3f-8813-40a705ece5c6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:37.055117 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-33b7edf2-ab68-4c3f-8813-40a705ece5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:37.076403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 701/1405] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:48:37.110675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6a14b-83b6-4d2b-bdbc-5794b8f74aa4 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:37.135656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6a14b-83b6-4d2b-bdbc-5794b8f74aa4 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:37.145146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177c2ac-70f0-4d15-9059-ab3a059f51fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:37.146109 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177c2ac-70f0-4d15-9059-ab3a059f51fc service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:37.146499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177c2ac-70f0-4d15-9059-ab3a059f51fc service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:37.146878 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177c2ac-70f0-4d15-9059-ab3a059f51fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:37.156766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177c2ac-70f0-4d15-9059-ab3a059f51fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:37.158439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177c2ac-70f0-4d15-9059-ab3a059f51fc service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:37.168072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 705/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:48:39.668231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708792d1-272c-4413-aaeb-3431ac781667 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:39.668788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708792d1-272c-4413-aaeb-3431ac781667 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:39.668788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708792d1-272c-4413-aaeb-3431ac781667 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:39.670120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 702/1407] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:48:39.676043 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c7643-2b51-49ab-9c1f-5fe59e678a50 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:39.676519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c7643-2b51-49ab-9c1f-5fe59e678a50 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:39.676643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c7643-2b51-49ab-9c1f-5fe59e678a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:39.695227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2a9c7643-2b51-49ab-9c1f-5fe59e678a50 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:48:39.696609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2a9c7643-2b51-49ab-9c1f-5fe59e678a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:48:39.706922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 706/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:48:39.752747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708792d1-272c-4413-aaeb-3431ac781667 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:48:39.768788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708792d1-272c-4413-aaeb-3431ac781667 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:48:39.774863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606995c8-fcdb-4938-856f-38482f977740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:48:39.775542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606995c8-fcdb-4938-856f-38482f977740 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:48:39.775825 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606995c8-fcdb-4938-856f-38482f977740 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:48:39.776099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606995c8-fcdb-4938-856f-38482f977740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:48:39.786178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606995c8-fcdb-4938-856f-38482f977740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:48:39.787800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606995c8-fcdb-4938-856f-38482f977740 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:48:39.810655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 703/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:48:49.690584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=54 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:48:49.793733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=23 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:01.936171 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54781b-f40f-43cb-a8fd-814c2d5dd846 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:01.936792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54781b-f40f-43cb-a8fd-814c2d5dd846 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:01.936792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54781b-f40f-43cb-a8fd-814c2d5dd846 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:01.938716 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 707/1410] 10.222.0.23 () {54 vars in 918 bytes} [Mon May 23 15:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:01.944070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f564d-8c36-4356-ba8f-7f40ad0ef715 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:01.944568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f564d-8c36-4356-ba8f-7f40ad0ef715 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:01.944672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f564d-8c36-4356-ba8f-7f40ad0ef715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:01.963037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-525f564d-8c36-4356-ba8f-7f40ad0ef715 None None] MFA Rules not processed for user `312ed2b555424c6a96b3d972ffea5ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:01.964264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-525f564d-8c36-4356-ba8f-7f40ad0ef715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:01.973348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 704/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Mon May 23 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:49:01.988788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1b058-4a0b-45ef-b5b6-fbfc8541f095 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:01.989081 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1b058-4a0b-45ef-b5b6-fbfc8541f095 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:01.989357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1b058-4a0b-45ef-b5b6-fbfc8541f095 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:01.990756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 708/1412] 10.222.0.23 () {54 vars in 918 bytes} [Mon May 23 15:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:01.995500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b3614-67e9-4b35-ac1c-35a711aea28c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:01.995824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b3614-67e9-4b35-ac1c-35a711aea28c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:01.996104 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b3614-67e9-4b35-ac1c-35a711aea28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:02.014670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6e5b3614-67e9-4b35-ac1c-35a711aea28c None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:02.015889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6e5b3614-67e9-4b35-ac1c-35a711aea28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:02.025128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 705/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Mon May 23 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:04.748237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48322baa-5b0c-4103-9c75-70f1eef80d77 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:04.748832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48322baa-5b0c-4103-9c75-70f1eef80d77 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:04.748832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48322baa-5b0c-4103-9c75-70f1eef80d77 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:04.750209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:04.755407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b4902-7d01-40e6-9675-3ca15a1f2dc9 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:04.755831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b4902-7d01-40e6-9675-3ca15a1f2dc9 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:04.755982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b4902-7d01-40e6-9675-3ca15a1f2dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:04.773916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e25b4902-7d01-40e6-9675-3ca15a1f2dc9 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:04.775091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e25b4902-7d01-40e6-9675-3ca15a1f2dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:04.783261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:04.824493 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48322baa-5b0c-4103-9c75-70f1eef80d77 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:04.838854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48322baa-5b0c-4103-9c75-70f1eef80d77 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:04.844668 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6119f03c-30e4-4c05-8f1c-9ca50f40f4bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:04.845294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119f03c-30e4-4c05-8f1c-9ca50f40f4bc service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:04.845574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119f03c-30e4-4c05-8f1c-9ca50f40f4bc service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:04.845844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119f03c-30e4-4c05-8f1c-9ca50f40f4bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:04.853779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6119f03c-30e4-4c05-8f1c-9ca50f40f4bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:04.855349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6119f03c-30e4-4c05-8f1c-9ca50f40f4bc service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:04.864543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 710/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:05.872278 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d50ad-8e5f-439a-8c48-d429571efb86 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:05.872905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d50ad-8e5f-439a-8c48-d429571efb86 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:05.872905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d50ad-8e5f-439a-8c48-d429571efb86 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:05.874233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:05.879286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9b35b-9dd6-474a-bcb8-3624c0ea9837 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:05.879720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9b35b-9dd6-474a-bcb8-3624c0ea9837 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:05.879862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9b35b-9dd6-474a-bcb8-3624c0ea9837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:05.898150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-81a9b35b-9dd6-474a-bcb8-3624c0ea9837 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:05.899360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-81a9b35b-9dd6-474a-bcb8-3624c0ea9837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:05.907741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:06.085753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d50ad-8e5f-439a-8c48-d429571efb86 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:06.101176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d50ad-8e5f-439a-8c48-d429571efb86 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:06.106690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca5123-e15b-45db-927a-cb4b04548941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:06.107446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca5123-e15b-45db-927a-cb4b04548941 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:06.107731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca5123-e15b-45db-927a-cb4b04548941 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:06.108044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca5123-e15b-45db-927a-cb4b04548941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:06.116818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca5123-e15b-45db-927a-cb4b04548941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:06.118410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca5123-e15b-45db-927a-cb4b04548941 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:06.128199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 708/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:07.095919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185a2fb-5602-4d39-ad68-972ee20d76b0 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:07.096409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185a2fb-5602-4d39-ad68-972ee20d76b0 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:07.096528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185a2fb-5602-4d39-ad68-972ee20d76b0 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:07.097875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:07.102847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a9b3a-6df6-4f62-a05b-6112653a812d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:07.103288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a9b3a-6df6-4f62-a05b-6112653a812d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:07.103388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a9b3a-6df6-4f62-a05b-6112653a812d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:07.121293 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-242a9b3a-6df6-4f62-a05b-6112653a812d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:07.122474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-242a9b3a-6df6-4f62-a05b-6112653a812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:07.130697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:07.277424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185a2fb-5602-4d39-ad68-972ee20d76b0 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:07.293019 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185a2fb-5602-4d39-ad68-972ee20d76b0 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:07.297892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecfddf-4f2c-4ba9-af04-6ee85384ccb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:07.298611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecfddf-4f2c-4ba9-af04-6ee85384ccb3 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:07.298941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecfddf-4f2c-4ba9-af04-6ee85384ccb3 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:07.299260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecfddf-4f2c-4ba9-af04-6ee85384ccb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:07.307747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ecfddf-4f2c-4ba9-af04-6ee85384ccb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:07.309358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ecfddf-4f2c-4ba9-af04-6ee85384ccb3 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:07.321517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 713/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:08.229333 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2ffa5-5a15-4e2d-974a-6d6597b697fc None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.229881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2ffa5-5a15-4e2d-974a-6d6597b697fc None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.229881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2ffa5-5a15-4e2d-974a-6d6597b697fc None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.231194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:08.236230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a02a9f-ccf8-4262-9b55-66e41bff2822 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.236690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a02a9f-ccf8-4262-9b55-66e41bff2822 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.236834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a02a9f-ccf8-4262-9b55-66e41bff2822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.257628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-40a02a9f-ccf8-4262-9b55-66e41bff2822 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:08.259367 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-40a02a9f-ccf8-4262-9b55-66e41bff2822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:08.267882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:08.416105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2ffa5-5a15-4e2d-974a-6d6597b697fc None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:08.430997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2ffa5-5a15-4e2d-974a-6d6597b697fc None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:08.436582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb507f-8a80-4360-9f29-7fd2b948037d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:08.437830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb507f-8a80-4360-9f29-7fd2b948037d service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:08.438143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb507f-8a80-4360-9f29-7fd2b948037d service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:08.438425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb507f-8a80-4360-9f29-7fd2b948037d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:08.447071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb507f-8a80-4360-9f29-7fd2b948037d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:08.448686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb507f-8a80-4360-9f29-7fd2b948037d service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:08.458140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 711/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:09.377447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe248de5-45ea-4900-942f-7b71785b888a None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:09.378000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe248de5-45ea-4900-942f-7b71785b888a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:09.378000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe248de5-45ea-4900-942f-7b71785b888a None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:09.379294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:09.384655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18ac84-5c24-40aa-8597-4b19b7829477 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:09.385064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18ac84-5c24-40aa-8597-4b19b7829477 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:09.385222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18ac84-5c24-40aa-8597-4b19b7829477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:09.403375 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1e18ac84-5c24-40aa-8597-4b19b7829477 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:09.404613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1e18ac84-5c24-40aa-8597-4b19b7829477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:09.413271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:09.560177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe248de5-45ea-4900-942f-7b71785b888a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:09.575150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe248de5-45ea-4900-942f-7b71785b888a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:09.581469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11398f96-07a7-47ab-9e6c-58bcfff9a718 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:09.582171 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11398f96-07a7-47ab-9e6c-58bcfff9a718 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:09.582576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11398f96-07a7-47ab-9e6c-58bcfff9a718 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:09.582873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11398f96-07a7-47ab-9e6c-58bcfff9a718 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:09.592116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11398f96-07a7-47ab-9e6c-58bcfff9a718 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:09.593949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11398f96-07a7-47ab-9e6c-58bcfff9a718 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:09.606227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 716/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:49:10.683782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452678e-fbd2-435f-8b32-f1b5cc33395e None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:10.684309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452678e-fbd2-435f-8b32-f1b5cc33395e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:10.684309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452678e-fbd2-435f-8b32-f1b5cc33395e None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:10.685691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:10.690693 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509752ac-3684-4bfe-bbf5-75ca28181a38 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:10.691096 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509752ac-3684-4bfe-bbf5-75ca28181a38 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:10.691251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509752ac-3684-4bfe-bbf5-75ca28181a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:10.710274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-509752ac-3684-4bfe-bbf5-75ca28181a38 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:10.711539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-509752ac-3684-4bfe-bbf5-75ca28181a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:10.720583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:10.870505 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452678e-fbd2-435f-8b32-f1b5cc33395e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:10.884979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452678e-fbd2-435f-8b32-f1b5cc33395e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:10.890790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745ecae-c92b-440e-a0d1-cd51d5c6dab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:10.891429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745ecae-c92b-440e-a0d1-cd51d5c6dab5 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:10.891710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745ecae-c92b-440e-a0d1-cd51d5c6dab5 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:10.891982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745ecae-c92b-440e-a0d1-cd51d5c6dab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:10.900057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745ecae-c92b-440e-a0d1-cd51d5c6dab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:10.901669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745ecae-c92b-440e-a0d1-cd51d5c6dab5 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:10.911668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 714/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:11.877129 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9baae6-e75a-414a-a903-442da9bd694b None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:11.877703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9baae6-e75a-414a-a903-442da9bd694b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:11.877703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9baae6-e75a-414a-a903-442da9bd694b None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:11.879054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:11.884380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d6ba5-3478-48e1-a132-fd5d918b5323 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:11.884798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d6ba5-3478-48e1-a132-fd5d918b5323 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:11.884983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d6ba5-3478-48e1-a132-fd5d918b5323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:11.902793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-419d6ba5-3478-48e1-a132-fd5d918b5323 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:11.903983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-419d6ba5-3478-48e1-a132-fd5d918b5323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:11.912302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:49:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:12.061490 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9baae6-e75a-414a-a903-442da9bd694b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:12.078596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9baae6-e75a-414a-a903-442da9bd694b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:12.084752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427101c-3f2b-414f-a081-1ee233e99718 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:12.085491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427101c-3f2b-414f-a081-1ee233e99718 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:12.085778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427101c-3f2b-414f-a081-1ee233e99718 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:12.086056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427101c-3f2b-414f-a081-1ee233e99718 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:12.096293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427101c-3f2b-414f-a081-1ee233e99718 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:12.098189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427101c-3f2b-414f-a081-1ee233e99718 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:12.109280 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 719/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:13.033273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e20dc-089c-4991-83ca-509f64c12bac None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:13.033840 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e20dc-089c-4991-83ca-509f64c12bac None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:13.033840 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e20dc-089c-4991-83ca-509f64c12bac None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:13.035211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:13.040438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf1d8b-34a1-4e2c-a232-62a849851fee None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:13.040868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf1d8b-34a1-4e2c-a232-62a849851fee None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:13.041022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf1d8b-34a1-4e2c-a232-62a849851fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:13.060268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-cbdf1d8b-34a1-4e2c-a232-62a849851fee None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:13.061574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-cbdf1d8b-34a1-4e2c-a232-62a849851fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:13.070362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:13.220007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e20dc-089c-4991-83ca-509f64c12bac None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:13.234914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e20dc-089c-4991-83ca-509f64c12bac None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:13.240344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7f940-39f1-4ae0-859a-4c536c75687f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:13.241011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7f940-39f1-4ae0-859a-4c536c75687f service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:13.241298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7f940-39f1-4ae0-859a-4c536c75687f service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:13.241578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7f940-39f1-4ae0-859a-4c536c75687f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:13.250536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7f940-39f1-4ae0-859a-4c536c75687f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:13.252115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7f940-39f1-4ae0-859a-4c536c75687f service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:13.261898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 717/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:14.163995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980b537-deca-4bfe-bc88-59b1800fb317 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:14.164570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980b537-deca-4bfe-bc88-59b1800fb317 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:14.164666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980b537-deca-4bfe-bc88-59b1800fb317 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:14.166093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:14.171271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634bc55-a0ad-4d72-94bf-ab3bd8d6a3c0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:14.172218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634bc55-a0ad-4d72-94bf-ab3bd8d6a3c0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:14.172429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634bc55-a0ad-4d72-94bf-ab3bd8d6a3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:14.190818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9634bc55-a0ad-4d72-94bf-ab3bd8d6a3c0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:14.192017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9634bc55-a0ad-4d72-94bf-ab3bd8d6a3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:14.200136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:49:14.347724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980b537-deca-4bfe-bc88-59b1800fb317 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:14.361907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980b537-deca-4bfe-bc88-59b1800fb317 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:14.367279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae06db2-7af2-4742-bda5-f10b3827e3e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:14.367916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae06db2-7af2-4742-bda5-f10b3827e3e5 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:14.368228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae06db2-7af2-4742-bda5-f10b3827e3e5 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:14.368554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae06db2-7af2-4742-bda5-f10b3827e3e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:14.376961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae06db2-7af2-4742-bda5-f10b3827e3e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:14.378593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae06db2-7af2-4742-bda5-f10b3827e3e5 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:14.388515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 722/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:16.501804 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be20811-a22c-4aba-bc66-fc761be6f640 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:16.502370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be20811-a22c-4aba-bc66-fc761be6f640 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:16.502370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be20811-a22c-4aba-bc66-fc761be6f640 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:16.503753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:16.509747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06853899-37fa-4ad3-8f7b-5bc6083b9eca None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:16.510237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06853899-37fa-4ad3-8f7b-5bc6083b9eca None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:16.510336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06853899-37fa-4ad3-8f7b-5bc6083b9eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:16.528405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-06853899-37fa-4ad3-8f7b-5bc6083b9eca None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:16.529587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-06853899-37fa-4ad3-8f7b-5bc6083b9eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:16.537946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:16.579226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be20811-a22c-4aba-bc66-fc761be6f640 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:16.590652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be20811-a22c-4aba-bc66-fc761be6f640 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:16.594909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4e11b-290a-44d7-93e7-dfb1069f6573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:16.595529 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4e11b-290a-44d7-93e7-dfb1069f6573 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:16.595813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4e11b-290a-44d7-93e7-dfb1069f6573 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:16.596096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4e11b-290a-44d7-93e7-dfb1069f6573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:16.605195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4e11b-290a-44d7-93e7-dfb1069f6573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:16.606624 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4e11b-290a-44d7-93e7-dfb1069f6573 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:16.616221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 720/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:18.958770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a5708-21ac-4db2-bc64-a807ac606030 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:18.959359 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a5708-21ac-4db2-bc64-a807ac606030 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:18.959359 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a5708-21ac-4db2-bc64-a807ac606030 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:18.960729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:18.966038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdfa90-6d85-417f-8c7d-15a72bfd6c05 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:18.966433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdfa90-6d85-417f-8c7d-15a72bfd6c05 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:18.966607 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdfa90-6d85-417f-8c7d-15a72bfd6c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:18.985313 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-40bdfa90-6d85-417f-8c7d-15a72bfd6c05 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:18.986638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-40bdfa90-6d85-417f-8c7d-15a72bfd6c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:18.994688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:19.033385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a5708-21ac-4db2-bc64-a807ac606030 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:19.044136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a5708-21ac-4db2-bc64-a807ac606030 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:19.047703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0fc247-a4de-4ce4-94b8-d4553d62645a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:19.048349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fc247-a4de-4ce4-94b8-d4553d62645a service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:19.048662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fc247-a4de-4ce4-94b8-d4553d62645a service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:19.048942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0fc247-a4de-4ce4-94b8-d4553d62645a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:19.058485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0fc247-a4de-4ce4-94b8-d4553d62645a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:19.059914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0fc247-a4de-4ce4-94b8-d4553d62645a service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:19.069651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 725/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:21.683478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28f99-bf25-47c2-a632-130b1a2c58f7 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:21.684052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28f99-bf25-47c2-a632-130b1a2c58f7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:21.684052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28f99-bf25-47c2-a632-130b1a2c58f7 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:21.685342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:21.690634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2af22b-f601-42fd-b29f-fc06fc61396b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:21.691093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2af22b-f601-42fd-b29f-fc06fc61396b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:21.691201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2af22b-f601-42fd-b29f-fc06fc61396b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:21.708663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2a2af22b-f601-42fd-b29f-fc06fc61396b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:21.709851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2a2af22b-f601-42fd-b29f-fc06fc61396b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:21.717522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:21.757105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd28f99-bf25-47c2-a632-130b1a2c58f7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:21.767990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd28f99-bf25-47c2-a632-130b1a2c58f7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:21.771472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2f958-2fa6-4b79-b3df-22c0f3d25139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:21.772105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2f958-2fa6-4b79-b3df-22c0f3d25139 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:21.772407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2f958-2fa6-4b79-b3df-22c0f3d25139 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:21.772688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2f958-2fa6-4b79-b3df-22c0f3d25139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:21.781932 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2f958-2fa6-4b79-b3df-22c0f3d25139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:21.783338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2f958-2fa6-4b79-b3df-22c0f3d25139 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:21.792697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 723/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:24.124058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129155d-1d25-4823-866a-77bdf3f72f6a None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:24.124672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129155d-1d25-4823-866a-77bdf3f72f6a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:24.124672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129155d-1d25-4823-866a-77bdf3f72f6a None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:24.126015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:24.131356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130e823-0867-4061-874a-cf0d0cfdb988 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:24.131768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130e823-0867-4061-874a-cf0d0cfdb988 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:24.131913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130e823-0867-4061-874a-cf0d0cfdb988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:24.149899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e130e823-0867-4061-874a-cf0d0cfdb988 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:24.151075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e130e823-0867-4061-874a-cf0d0cfdb988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:24.159249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:24.196517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129155d-1d25-4823-866a-77bdf3f72f6a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:24.207655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129155d-1d25-4823-866a-77bdf3f72f6a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:24.211416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b80069-65c6-45f3-bc3d-85bcf63a5f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:24.212062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b80069-65c6-45f3-bc3d-85bcf63a5f40 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:24.212351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b80069-65c6-45f3-bc3d-85bcf63a5f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:24.212682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b80069-65c6-45f3-bc3d-85bcf63a5f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:24.222196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b80069-65c6-45f3-bc3d-85bcf63a5f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:24.223625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b80069-65c6-45f3-bc3d-85bcf63a5f40 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:24.232992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 728/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:26.565059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52919b9-6494-4fbc-8ced-21554b7bda15 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:26.565580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52919b9-6494-4fbc-8ced-21554b7bda15 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:26.565580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52919b9-6494-4fbc-8ced-21554b7bda15 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:26.566943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:26.572513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35336cb4-a73b-43cf-a510-aaf242dfe2b1 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:26.573033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35336cb4-a73b-43cf-a510-aaf242dfe2b1 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:26.573131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35336cb4-a73b-43cf-a510-aaf242dfe2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:26.590775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-35336cb4-a73b-43cf-a510-aaf242dfe2b1 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:26.591955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-35336cb4-a73b-43cf-a510-aaf242dfe2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:26.599514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:26.637868 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52919b9-6494-4fbc-8ced-21554b7bda15 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:26.648827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52919b9-6494-4fbc-8ced-21554b7bda15 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:26.652484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fe4d4-80f9-440e-b58b-a7d19fdf1d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:26.653118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe4d4-80f9-440e-b58b-a7d19fdf1d49 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:26.653399 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe4d4-80f9-440e-b58b-a7d19fdf1d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:26.653674 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe4d4-80f9-440e-b58b-a7d19fdf1d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:26.662425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fe4d4-80f9-440e-b58b-a7d19fdf1d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:26.663838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fe4d4-80f9-440e-b58b-a7d19fdf1d49 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:26.673528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 726/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:28.944779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16889e-7732-40d3-afe3-074c9567a14e None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:28.945345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16889e-7732-40d3-afe3-074c9567a14e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:28.945345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16889e-7732-40d3-afe3-074c9567a14e None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:28.946775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:28.952107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb3427-2991-4b90-9f51-ec35f4c6a64c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:28.952552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb3427-2991-4b90-9f51-ec35f4c6a64c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:28.952690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb3427-2991-4b90-9f51-ec35f4c6a64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:28.970189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-fadb3427-2991-4b90-9f51-ec35f4c6a64c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:28.971382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-fadb3427-2991-4b90-9f51-ec35f4c6a64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:28.979073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:29.016862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16889e-7732-40d3-afe3-074c9567a14e None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:29.027719 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16889e-7732-40d3-afe3-074c9567a14e None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:29.031228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef8d4d-a5b7-4a9f-973f-daee77e0fa22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:29.031864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef8d4d-a5b7-4a9f-973f-daee77e0fa22 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:29.032146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef8d4d-a5b7-4a9f-973f-daee77e0fa22 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:29.032456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef8d4d-a5b7-4a9f-973f-daee77e0fa22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:29.041387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef8d4d-a5b7-4a9f-973f-daee77e0fa22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:29.042843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef8d4d-a5b7-4a9f-973f-daee77e0fa22 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:29.052697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 731/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:31.371732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5173725-cd7d-4a96-987b-8e385e01de58 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:31.372240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5173725-cd7d-4a96-987b-8e385e01de58 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:31.372347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5173725-cd7d-4a96-987b-8e385e01de58 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:31.373750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:31.379762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3987-b06f-458b-a3fb-56acb2344665 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:31.380175 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3987-b06f-458b-a3fb-56acb2344665 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:31.380295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3987-b06f-458b-a3fb-56acb2344665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:31.386600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=70 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:31.399027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-197e3987-b06f-458b-a3fb-56acb2344665 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:31.400218 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-197e3987-b06f-458b-a3fb-56acb2344665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:31.408191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:49:31.445808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5173725-cd7d-4a96-987b-8e385e01de58 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:31.451221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=73 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:31.457509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5173725-cd7d-4a96-987b-8e385e01de58 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:31.460997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b2131-6961-4ae0-a0c8-1d97dc9f4521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:31.461662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b2131-6961-4ae0-a0c8-1d97dc9f4521 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:31.461946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b2131-6961-4ae0-a0c8-1d97dc9f4521 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:31.462221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b2131-6961-4ae0-a0c8-1d97dc9f4521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:31.471337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b2131-6961-4ae0-a0c8-1d97dc9f4521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:31.472799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b2131-6961-4ae0-a0c8-1d97dc9f4521 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:31.482513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 729/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:49:33.765638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0cc16-d590-4d77-821f-5a32eec711a1 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:33.766264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0cc16-d590-4d77-821f-5a32eec711a1 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:33.766264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0cc16-d590-4d77-821f-5a32eec711a1 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:33.767558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:33.772966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa28d7f-e974-4e14-9932-4a59770d19b6 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:33.773410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa28d7f-e974-4e14-9932-4a59770d19b6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:33.773508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa28d7f-e974-4e14-9932-4a59770d19b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:33.791188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-dfa28d7f-e974-4e14-9932-4a59770d19b6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:33.792392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dfa28d7f-e974-4e14-9932-4a59770d19b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:33.800792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:49:33.837417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0cc16-d590-4d77-821f-5a32eec711a1 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:33.850849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0cc16-d590-4d77-821f-5a32eec711a1 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:33.854860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53182fd-65de-4867-a94d-4f5b8873282b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:33.855503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53182fd-65de-4867-a94d-4f5b8873282b service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:33.855789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53182fd-65de-4867-a94d-4f5b8873282b service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:33.856069 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53182fd-65de-4867-a94d-4f5b8873282b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:33.866740 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53182fd-65de-4867-a94d-4f5b8873282b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:33.868174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53182fd-65de-4867-a94d-4f5b8873282b service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:33.877774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 734/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:36.105166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c5bfb-197b-4822-97e3-b9fd105ab16d None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:36.105761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c5bfb-197b-4822-97e3-b9fd105ab16d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:36.105761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c5bfb-197b-4822-97e3-b9fd105ab16d None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:36.107249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:36.113219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35fd48-c76c-4718-8aa0-074507238b7d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:36.113665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35fd48-c76c-4718-8aa0-074507238b7d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:36.113882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35fd48-c76c-4718-8aa0-074507238b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:36.132976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-cd35fd48-c76c-4718-8aa0-074507238b7d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:36.134424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-cd35fd48-c76c-4718-8aa0-074507238b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:36.142278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:36.178707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c5bfb-197b-4822-97e3-b9fd105ab16d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:36.189412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c5bfb-197b-4822-97e3-b9fd105ab16d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:36.192873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68268b3-6363-4f63-8bf3-93ce0a135a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:36.193508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68268b3-6363-4f63-8bf3-93ce0a135a47 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:36.193788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68268b3-6363-4f63-8bf3-93ce0a135a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:36.194063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68268b3-6363-4f63-8bf3-93ce0a135a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:36.202773 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68268b3-6363-4f63-8bf3-93ce0a135a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:36.204188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68268b3-6363-4f63-8bf3-93ce0a135a47 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:36.213382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 732/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:38.410008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbde833-8c8e-46e2-83e1-0c41a15444d3 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:38.410549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbde833-8c8e-46e2-83e1-0c41a15444d3 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:38.410549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbde833-8c8e-46e2-83e1-0c41a15444d3 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:38.411848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:49:38.417138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd2879-9a98-4f49-b2d8-ef1dd02fa30d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:38.417539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd2879-9a98-4f49-b2d8-ef1dd02fa30d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:38.417729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd2879-9a98-4f49-b2d8-ef1dd02fa30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:38.435681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-44dd2879-9a98-4f49-b2d8-ef1dd02fa30d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:49:38.436891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-44dd2879-9a98-4f49-b2d8-ef1dd02fa30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:49:38.444519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:49:38.481117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbde833-8c8e-46e2-83e1-0c41a15444d3 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:49:38.491822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbde833-8c8e-46e2-83e1-0c41a15444d3 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:49:38.495341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1b420-b5dc-4194-8d5f-1ae56c4352c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:49:38.495968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1b420-b5dc-4194-8d5f-1ae56c4352c7 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:49:38.496248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1b420-b5dc-4194-8d5f-1ae56c4352c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:49:38.496548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1b420-b5dc-4194-8d5f-1ae56c4352c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:49:38.505781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1b420-b5dc-4194-8d5f-1ae56c4352c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:49:38.507364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1b420-b5dc-4194-8d5f-1ae56c4352c7 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:49:38.517949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 737/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:49:48.430720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=17 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:49:48.513262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=17 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:12.303973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee397f-4bf2-4e72-9edb-1beec302ee67 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:12.304551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee397f-4bf2-4e72-9edb-1beec302ee67 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:12.304551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee397f-4bf2-4e72-9edb-1beec302ee67 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:12.305957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 734/1471] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:50:12.311068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed689e-40fa-42c9-8708-709b1dc64b0c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:12.311511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed689e-40fa-42c9-8708-709b1dc64b0c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:12.311620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed689e-40fa-42c9-8708-709b1dc64b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:12.330339 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-6eed689e-40fa-42c9-8708-709b1dc64b0c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:12.331564 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6eed689e-40fa-42c9-8708-709b1dc64b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:12.339353 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 738/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:12.385959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050b3d-940b-45d8-90f7-976627fd8f8b None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:12.386266 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050b3d-940b-45d8-90f7-976627fd8f8b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:12.386548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050b3d-940b-45d8-90f7-976627fd8f8b None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:12.387949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 735/1473] 10.222.0.23 () {54 vars in 962 bytes} [Mon May 23 15:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:50:12.392796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5315c-5d80-4d16-8ac7-af49a272ab77 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:12.393091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5315c-5d80-4d16-8ac7-af49a272ab77 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:12.393371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5315c-5d80-4d16-8ac7-af49a272ab77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:12.411334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-04a5315c-5d80-4d16-8ac7-af49a272ab77 None None] MFA Rules not processed for user `142e6e44d6544f6a891ed3f913de74ae`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:12.412563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-04a5315c-5d80-4d16-8ac7-af49a272ab77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:12.419151 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 739/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Mon May 23 15:50:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:12.427057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050b3d-940b-45d8-90f7-976627fd8f8b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:12.437523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a050b3d-940b-45d8-90f7-976627fd8f8b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:12.441756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9ab46-e201-413f-ad15-8ac886bae8fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '142e6e44d6544f6a891ed3f913de74ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:12.442386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9ab46-e201-413f-ad15-8ac886bae8fc service glance] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:12.442668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9ab46-e201-413f-ad15-8ac886bae8fc service glance] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:12.442943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9ab46-e201-413f-ad15-8ac886bae8fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:12.451506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9ab46-e201-413f-ad15-8ac886bae8fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:12.453003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9ab46-e201-413f-ad15-8ac886bae8fc service glance] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:12.462245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 736/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:50:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:14.771061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6cc2a-ac20-486a-beaf-6454d421a16a None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:14.771651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6cc2a-ac20-486a-beaf-6454d421a16a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:14.771651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6cc2a-ac20-486a-beaf-6454d421a16a None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:14.772991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 740/1476] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:50:14.778115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132eeab6-18f5-4fda-9674-9afb89bb9f47 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:14.778551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132eeab6-18f5-4fda-9674-9afb89bb9f47 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:14.778706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132eeab6-18f5-4fda-9674-9afb89bb9f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:14.796190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-132eeab6-18f5-4fda-9674-9afb89bb9f47 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:14.797920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-132eeab6-18f5-4fda-9674-9afb89bb9f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:14.805928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 737/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:14.829379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6cc2a-ac20-486a-beaf-6454d421a16a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:14.841981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6cc2a-ac20-486a-beaf-6454d421a16a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:14.846879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec575f-8add-47a4-9e7b-64cc2564edeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:14.847555 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec575f-8add-47a4-9e7b-64cc2564edeb admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:14.847843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec575f-8add-47a4-9e7b-64cc2564edeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:14.848130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec575f-8add-47a4-9e7b-64cc2564edeb admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:14.848891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec575f-8add-47a4-9e7b-64cc2564edeb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:14.850683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WARNING py.warnings [None req-8bec575f-8add-47a4-9e7b-64cc2564edeb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:50:14.850683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: warnings.warn(msg) May 23 15:50:14.850683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: May 23 15:50:14.851804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec575f-8add-47a4-9e7b-64cc2564edeb admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:14.871394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 741/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Mon May 23 15:50:14 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:50:17.094087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd76dc-9bc1-427b-8173-9282e8ee73cd None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:17.095268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd76dc-9bc1-427b-8173-9282e8ee73cd None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:17.095268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd76dc-9bc1-427b-8173-9282e8ee73cd None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:17.096598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 738/1479] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:50:17.102197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6989706-2c68-4720-bd5e-788d00cbbe35 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:17.102578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6989706-2c68-4720-bd5e-788d00cbbe35 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:17.102757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6989706-2c68-4720-bd5e-788d00cbbe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:17.120253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-c6989706-2c68-4720-bd5e-788d00cbbe35 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:17.121565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c6989706-2c68-4720-bd5e-788d00cbbe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:17.129776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 742/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:17.173745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd76dc-9bc1-427b-8173-9282e8ee73cd None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:17.187888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd76dc-9bc1-427b-8173-9282e8ee73cd None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:17.193610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c6229-9ed6-40a0-a1a2-22254a4826da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:17.194237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c6229-9ed6-40a0-a1a2-22254a4826da service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:17.194514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c6229-9ed6-40a0-a1a2-22254a4826da service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:17.194785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c6229-9ed6-40a0-a1a2-22254a4826da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:17.202720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c6229-9ed6-40a0-a1a2-22254a4826da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:17.204960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c6229-9ed6-40a0-a1a2-22254a4826da service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:17.214081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 739/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:19.610714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bd5b3-ac07-4c87-94d0-2ed4716a04af None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:19.611337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bd5b3-ac07-4c87-94d0-2ed4716a04af None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:19.611337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bd5b3-ac07-4c87-94d0-2ed4716a04af None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:19.612835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 743/1482] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:50:19.618046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55446774-8266-48a8-aa97-9f9b8a48c096 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:19.618528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55446774-8266-48a8-aa97-9f9b8a48c096 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:19.618623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55446774-8266-48a8-aa97-9f9b8a48c096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:19.636787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-55446774-8266-48a8-aa97-9f9b8a48c096 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:19.637993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-55446774-8266-48a8-aa97-9f9b8a48c096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:19.646317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 740/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:19.681443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bd5b3-ac07-4c87-94d0-2ed4716a04af None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:19.692549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bd5b3-ac07-4c87-94d0-2ed4716a04af None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:19.695961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f5b1e-cfa9-4863-b673-9c50e718c5c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:19.696628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f5b1e-cfa9-4863-b673-9c50e718c5c0 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:19.696913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f5b1e-cfa9-4863-b673-9c50e718c5c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:19.697189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f5b1e-cfa9-4863-b673-9c50e718c5c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:19.706352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f5b1e-cfa9-4863-b673-9c50e718c5c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:19.707758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f5b1e-cfa9-4863-b673-9c50e718c5c0 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:19.717042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 744/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:50:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:50:21.976823 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82797f-cfa7-4d43-be30-4ed178a38d21 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:21.986354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82797f-cfa7-4d43-be30-4ed178a38d21 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:21.986354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82797f-cfa7-4d43-be30-4ed178a38d21 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:21.986354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 741/1485] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:50:21.986672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe65ca-3e14-4588-acc8-f061eb582661 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:21.986672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe65ca-3e14-4588-acc8-f061eb582661 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:21.986672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe65ca-3e14-4588-acc8-f061eb582661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:22.002482 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-38fe65ca-3e14-4588-acc8-f061eb582661 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:22.003651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-38fe65ca-3e14-4588-acc8-f061eb582661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:22.011665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 745/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:50:22.049381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82797f-cfa7-4d43-be30-4ed178a38d21 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:22.059990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82797f-cfa7-4d43-be30-4ed178a38d21 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:22.063527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e9f4b-4747-4668-b2a9-f0fc0a47fc80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:22.064162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e9f4b-4747-4668-b2a9-f0fc0a47fc80 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:22.064485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e9f4b-4747-4668-b2a9-f0fc0a47fc80 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:22.064767 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e9f4b-4747-4668-b2a9-f0fc0a47fc80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:22.073535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e9f4b-4747-4668-b2a9-f0fc0a47fc80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:22.074958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e9f4b-4747-4668-b2a9-f0fc0a47fc80 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:22.084780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 742/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:50:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:24.515824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5b51a-6b8a-4e58-ac44-37fa970bcc93 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:24.516891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5b51a-6b8a-4e58-ac44-37fa970bcc93 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:24.516891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5b51a-6b8a-4e58-ac44-37fa970bcc93 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:24.518293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 746/1488] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:50:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:50:24.523724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf55a-9452-411a-a92d-747996dc69ff None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:24.524185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf55a-9452-411a-a92d-747996dc69ff None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:24.524310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf55a-9452-411a-a92d-747996dc69ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:24.542445 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-966bf55a-9452-411a-a92d-747996dc69ff None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:24.543563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-966bf55a-9452-411a-a92d-747996dc69ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:24.551487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 743/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:24.591248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5b51a-6b8a-4e58-ac44-37fa970bcc93 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:24.602566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5b51a-6b8a-4e58-ac44-37fa970bcc93 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:24.606020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95277020-bcab-4db0-a935-80909e41eca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:24.606652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95277020-bcab-4db0-a935-80909e41eca9 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:24.606936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95277020-bcab-4db0-a935-80909e41eca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:24.607212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95277020-bcab-4db0-a935-80909e41eca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:24.616597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95277020-bcab-4db0-a935-80909e41eca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:24.618022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95277020-bcab-4db0-a935-80909e41eca9 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:24.627863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 747/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:50:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:27.349084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03747c-96a1-4cfb-8b55-7b01c1cbfeab None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:27.349638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03747c-96a1-4cfb-8b55-7b01c1cbfeab None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:27.349638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03747c-96a1-4cfb-8b55-7b01c1cbfeab None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:27.351022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 744/1491] 10.222.0.23 () {54 vars in 926 bytes} [Mon May 23 15:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:50:27.356413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f8c0f-7674-4bb2-95a7-b020dd61d0b3 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:27.356842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f8c0f-7674-4bb2-95a7-b020dd61d0b3 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:27.356996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f8c0f-7674-4bb2-95a7-b020dd61d0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:27.375194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-9b9f8c0f-7674-4bb2-95a7-b020dd61d0b3 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:50:27.377050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-9b9f8c0f-7674-4bb2-95a7-b020dd61d0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:50:27.384706 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 748/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Mon May 23 15:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:50:27.423547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03747c-96a1-4cfb-8b55-7b01c1cbfeab None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:50:27.434095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03747c-96a1-4cfb-8b55-7b01c1cbfeab None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:50:27.439425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e1c70-18dc-4900-a907-775b504ef748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:50:27.440477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e1c70-18dc-4900-a907-775b504ef748 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:50:27.440954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e1c70-18dc-4900-a907-775b504ef748 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:50:27.441419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e1c70-18dc-4900-a907-775b504ef748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:50:27.451687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e1c70-18dc-4900-a907-775b504ef748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:50:27.453127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e1c70-18dc-4900-a907-775b504ef748 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:50:27.462327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 745/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:50:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:50:37.418420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=33 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:50:37.458200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=33 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:19.607228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c9ef2-74d8-43c5-8ded-13387867d624 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:19.607835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c9ef2-74d8-43c5-8ded-13387867d624 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:19.607835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c9ef2-74d8-43c5-8ded-13387867d624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:19.625312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e97c9ef2-74d8-43c5-8ded-13387867d624 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:19.626500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e97c9ef2-74d8-43c5-8ded-13387867d624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:19.634226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 749/1494] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:19.644012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd5f4b-a6e2-4fe4-aaeb-0c1971deb333 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:19.645144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd5f4b-a6e2-4fe4-aaeb-0c1971deb333 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:19.645144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd5f4b-a6e2-4fe4-aaeb-0c1971deb333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:19.665203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-20fd5f4b-a6e2-4fe4-aaeb-0c1971deb333 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:19.666494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-20fd5f4b-a6e2-4fe4-aaeb-0c1971deb333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:19.674695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 746/1495] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:19.679588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c9ef2-74d8-43c5-8ded-13387867d624 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:19.689415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c9ef2-74d8-43c5-8ded-13387867d624 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:19.694039 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2478cf-6be4-4fda-9578-8df8068ffb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:19.694722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2478cf-6be4-4fda-9578-8df8068ffb01 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:19.695002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2478cf-6be4-4fda-9578-8df8068ffb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:19.695289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2478cf-6be4-4fda-9578-8df8068ffb01 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:19.695992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2478cf-6be4-4fda-9578-8df8068ffb01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:52:19.696302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2478cf-6be4-4fda-9578-8df8068ffb01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:19.697346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WARNING py.warnings [None req-cf2478cf-6be4-4fda-9578-8df8068ffb01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:52:19.697346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: warnings.warn(msg) May 23 15:52:19.697346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: May 23 15:52:19.698209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2478cf-6be4-4fda-9578-8df8068ffb01 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:19.710460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 750/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:52:19.718667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd5f4b-a6e2-4fe4-aaeb-0c1971deb333 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:19.730845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd5f4b-a6e2-4fe4-aaeb-0c1971deb333 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:19.735340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ae08-020b-425c-a7e8-69959cfacf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:19.735999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ae08-020b-425c-a7e8-69959cfacf2e admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:19.736279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ae08-020b-425c-a7e8-69959cfacf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:19.736595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ae08-020b-425c-a7e8-69959cfacf2e admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:19.737281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111ae08-020b-425c-a7e8-69959cfacf2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:19.739043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: WARNING py.warnings [None req-6111ae08-020b-425c-a7e8-69959cfacf2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:52:19.739043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: warnings.warn(msg) May 23 15:52:19.739043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: May 23 15:52:19.740235 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111ae08-020b-425c-a7e8-69959cfacf2e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:19.870373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 747/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:52:19 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:52:19.874611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2478cf-6be4-4fda-9578-8df8068ffb01 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:19.885123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2478cf-6be4-4fda-9578-8df8068ffb01 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:19.940066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d351983-4620-4006-bbb6-27531d8f08a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:19.940791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d351983-4620-4006-bbb6-27531d8f08a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:19.941621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d351983-4620-4006-bbb6-27531d8f08a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:19.941916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d351983-4620-4006-bbb6-27531d8f08a9 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:19.942703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d351983-4620-4006-bbb6-27531d8f08a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:19.943746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WARNING py.warnings [None req-9d351983-4620-4006-bbb6-27531d8f08a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:52:19.943746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: warnings.warn(msg) May 23 15:52:19.943746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: May 23 15:52:19.944978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d351983-4620-4006-bbb6-27531d8f08a9 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:20.092961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 751/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:52:19 2022] POST /identity/v3/users => generated 290 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:52:20.097208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ae08-020b-425c-a7e8-69959cfacf2e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:20.108217 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ae08-020b-425c-a7e8-69959cfacf2e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:20.114103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b267501-caba-4c5b-819f-bd3d8b551d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:20.114897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b267501-caba-4c5b-819f-bd3d8b551d2d admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:20.115181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b267501-caba-4c5b-819f-bd3d8b551d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:20.115456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b267501-caba-4c5b-819f-bd3d8b551d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:20.116190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b267501-caba-4c5b-819f-bd3d8b551d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:20.117243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: WARNING py.warnings [None req-1b267501-caba-4c5b-819f-bd3d8b551d2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:52:20.117243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: warnings.warn(msg) May 23 15:52:20.117243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: May 23 15:52:20.118134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b267501-caba-4c5b-819f-bd3d8b551d2d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:20.131771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 748/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:52:20.140149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d351983-4620-4006-bbb6-27531d8f08a9 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:20.150793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d351983-4620-4006-bbb6-27531d8f08a9 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:20.155371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42461d03-f3c6-4414-9f12-27ddea722145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:20.156043 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42461d03-f3c6-4414-9f12-27ddea722145 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:20.156329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42461d03-f3c6-4414-9f12-27ddea722145 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:20.156635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42461d03-f3c6-4414-9f12-27ddea722145 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:20.157435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42461d03-f3c6-4414-9f12-27ddea722145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:20.158422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WARNING py.warnings [None req-42461d03-f3c6-4414-9f12-27ddea722145 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:52:20.158422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: warnings.warn(msg) May 23 15:52:20.158422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: May 23 15:52:20.159309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42461d03-f3c6-4414-9f12-27ddea722145 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:20.173515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 752/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:52:20.177826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b267501-caba-4c5b-819f-bd3d8b551d2d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:20.188106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b267501-caba-4c5b-819f-bd3d8b551d2d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:20.192739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9daff-6c18-467c-a897-50faf7c050ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:20.193668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9daff-6c18-467c-a897-50faf7c050ad admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:20.193958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9daff-6c18-467c-a897-50faf7c050ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:20.194235 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9daff-6c18-467c-a897-50faf7c050ad admin admin] PATH_INFO: `/v3/projects/fb250897d80441e0842dd782f0cfe674/users/c7d9602dc23342b795384ebba84c3c98/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:20.209539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9daff-6c18-467c-a897-50faf7c050ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb250897d80441e0842dd782f0cfe674, user_id=c7d9602dc23342b795384ebba84c3c98, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:20.210573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: WARNING py.warnings [None req-84b9daff-6c18-467c-a897-50faf7c050ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:52:20.210573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: warnings.warn(msg) May 23 15:52:20.210573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: May 23 15:52:20.212561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9daff-6c18-467c-a897-50faf7c050ad admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:20.257223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 749/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:52:20 2022] PUT /identity/v3/projects/fb250897d80441e0842dd782f0cfe674/users/c7d9602dc23342b795384ebba84c3c98/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:52:20.274558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5774dd-6d0e-4e9c-8aee-3c3f0e291a68 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:20.274893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5774dd-6d0e-4e9c-8aee-3c3f0e291a68 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:20.275174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5774dd-6d0e-4e9c-8aee-3c3f0e291a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:20.303062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0c5774dd-6d0e-4e9c-8aee-3c3f0e291a68 None None] MFA Rules not processed for user `c7d9602dc23342b795384ebba84c3c98`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:20.304285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0c5774dd-6d0e-4e9c-8aee-3c3f0e291a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:20.367914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 753/1502] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:52:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:20.381368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4957e6-4044-402e-b55e-6d321d1d7236 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:20.381663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4957e6-4044-402e-b55e-6d321d1d7236 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:20.382001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4957e6-4044-402e-b55e-6d321d1d7236 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:20.383326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 750/1503] 10.222.0.23 () {54 vars in 963 bytes} [Mon May 23 15:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:52:20.388846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2acf4-f8bf-4c9e-9838-59fb08360734 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:20.389142 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2acf4-f8bf-4c9e-9838-59fb08360734 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:20.389418 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2acf4-f8bf-4c9e-9838-59fb08360734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:20.409054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-3ae2acf4-f8bf-4c9e-9838-59fb08360734 None None] MFA Rules not processed for user `0d30c8d1f883400ba397ea09d42a0aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:20.410840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3ae2acf4-f8bf-4c9e-9838-59fb08360734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:20.453861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:20.461892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 754/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Mon May 23 15:52:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:20.574581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4957e6-4044-402e-b55e-6d321d1d7236 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:20.585213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4957e6-4044-402e-b55e-6d321d1d7236 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:20.588831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dd111-6e6f-42ac-b490-d9243f8e20f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:20.589476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dd111-6e6f-42ac-b490-d9243f8e20f6 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:20.590622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dd111-6e6f-42ac-b490-d9243f8e20f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:20.590622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dd111-6e6f-42ac-b490-d9243f8e20f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:20.599118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dd111-6e6f-42ac-b490-d9243f8e20f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:20.600673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dd111-6e6f-42ac-b490-d9243f8e20f6 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:20.609568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 751/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:52:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:52:20.652431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f6023-916c-437f-b8e4-fb53b53740cb None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:20.652907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f6023-916c-437f-b8e4-fb53b53740cb None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:20.653000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f6023-916c-437f-b8e4-fb53b53740cb None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:20.654273 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 755/1506] 10.222.0.23 () {52 vars in 829 bytes} [Mon May 23 15:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:52:20.657951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dd111-6e6f-42ac-b490-d9243f8e20f6 service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:20.719246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dd111-6e6f-42ac-b490-d9243f8e20f6 service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:20.724307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:20.725201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:20.725481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:20.725753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] PATH_INFO: `/v3/users/c7d9602dc23342b795384ebba84c3c98` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:20.726923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7d9602dc23342b795384ebba84c3c98)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:20.727920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: WARNING py.warnings [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:52:20.727920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: warnings.warn(msg) May 23 15:52:20.727920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: May 23 15:52:20.729209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:21.169393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7d9602dc23342b795384ebba84c3c98'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:21.170030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7d9602dc23342b795384ebba84c3c98'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:21.188450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7d9602dc23342b795384ebba84c3c98'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:21.189219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7d9602dc23342b795384ebba84c3c98'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:21.201619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7d9602dc23342b795384ebba84c3c98'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:21.205561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:21.319416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 752/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:52:20 2022] DELETE /identity/v3/users/c7d9602dc23342b795384ebba84c3c98 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:52:21.323849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19850c-0355-4208-b55b-c788867bb2cf None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:21.324194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19850c-0355-4208-b55b-c788867bb2cf None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:21.324529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19850c-0355-4208-b55b-c788867bb2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:21.342700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-7a19850c-0355-4208-b55b-c788867bb2cf None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:52:21.345220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7a19850c-0355-4208-b55b-c788867bb2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:21.403588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 756/1508] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:52:21.413189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:21.414641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:21.471781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ec6c6-7ff4-4934-bdfa-10a55d873a46 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:21.475272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a89da-6f8d-4139-9f60-ac8972a2a47f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:21.475920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a89da-6f8d-4139-9f60-ac8972a2a47f service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:21.476202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a89da-6f8d-4139-9f60-ac8972a2a47f service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:21.476509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a89da-6f8d-4139-9f60-ac8972a2a47f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:21.485318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a89da-6f8d-4139-9f60-ac8972a2a47f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:21.486747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a89da-6f8d-4139-9f60-ac8972a2a47f service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:21.496055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 753/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:52:21.974039 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19850c-0355-4208-b55b-c788867bb2cf None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:52:21.975475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7a19850c-0355-4208-b55b-c788867bb2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:52:21.992825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19850c-0355-4208-b55b-c788867bb2cf None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:52:21.994801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:52:21.996154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:52:21.996610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:52:21.996735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] PATH_INFO: `/v3/projects/fb250897d80441e0842dd782f0cfe674` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:52:21.998673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb250897d80441e0842dd782f0cfe674)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:52:21.999915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WARNING py.warnings [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:52:21.999915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: warnings.warn(msg) May 23 15:52:21.999915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: May 23 15:52:22.001132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:52:22.130064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb250897d80441e0842dd782f0cfe674', 'request_id': 'req-c66ffb52-5546-48ca-bd5b-741fcbd67c35'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:22.148450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb250897d80441e0842dd782f0cfe674', 'request_id': 'req-c66ffb52-5546-48ca-bd5b-741fcbd67c35'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:22.154986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb250897d80441e0842dd782f0cfe674', 'request_id': 'req-c66ffb52-5546-48ca-bd5b-741fcbd67c35'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:22.257246 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] The token cache is being invalidate because project fb250897d80441e0842dd782f0cfe674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:52:22.257540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:22.258185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c66ffb52-5546-48ca-bd5b-741fcbd67c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:52:22.259829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 757/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:52:21 2022] DELETE /identity/v3/projects/fb250897d80441e0842dd782f0cfe674 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:52:31.491977 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:52:32.253980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:03.616637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e2619-d0e5-41b4-a2cd-453c6f976a08 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:03.619283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e2619-d0e5-41b4-a2cd-453c6f976a08 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:03.619283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e2619-d0e5-41b4-a2cd-453c6f976a08 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:03.619484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:54:03.624057 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799a509-10dd-45ca-9343-92ee32c220c0 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:03.624492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799a509-10dd-45ca-9343-92ee32c220c0 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:03.624683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799a509-10dd-45ca-9343-92ee32c220c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:03.642776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e799a509-10dd-45ca-9343-92ee32c220c0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:03.647523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e799a509-10dd-45ca-9343-92ee32c220c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:03.701643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:03.874466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e2619-d0e5-41b4-a2cd-453c6f976a08 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:03.876241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1e7e2619-d0e5-41b4-a2cd-453c6f976a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:03.952746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e2619-d0e5-41b4-a2cd-453c6f976a08 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:03.958930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284528be-c215-4055-a6ba-f50418d27620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:03.959664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284528be-c215-4055-a6ba-f50418d27620 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:03.959983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284528be-c215-4055-a6ba-f50418d27620 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:03.960371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284528be-c215-4055-a6ba-f50418d27620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:03.970188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284528be-c215-4055-a6ba-f50418d27620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:03.971946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284528be-c215-4055-a6ba-f50418d27620 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:03.984509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 755/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:54:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:04.057998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bc027-c4ab-4bca-a000-6b7f8193cf7e None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:04.058379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bc027-c4ab-4bca-a000-6b7f8193cf7e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:04.058729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bc027-c4ab-4bca-a000-6b7f8193cf7e None None] PATH_INFO: `/v3` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:04.060635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 759/1514] 10.222.0.50 () {54 vars in 925 bytes} [Mon May 23 15:54:04 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:54:04.066793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b544850-0bbc-4065-a317-8d3175f62b3f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:04.067097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b544850-0bbc-4065-a317-8d3175f62b3f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:04.067369 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b544850-0bbc-4065-a317-8d3175f62b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:04.086941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2b544850-0bbc-4065-a317-8d3175f62b3f None None] MFA Rules not processed for user `312ed2b555424c6a96b3d972ffea5ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:04.089585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2b544850-0bbc-4065-a317-8d3175f62b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:04.157859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 756/1515] 10.222.0.50 () {58 vars in 1015 bytes} [Mon May 23 15:54:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:04.865974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53738e79-ce43-4913-91ab-8f6a021e99ab None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:04.866556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53738e79-ce43-4913-91ab-8f6a021e99ab None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:04.866556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53738e79-ce43-4913-91ab-8f6a021e99ab None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:04.868001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 760/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:54:04.873466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9a391-dd86-4b50-8598-72f59f04d193 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:04.873903 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9a391-dd86-4b50-8598-72f59f04d193 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:04.874033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9a391-dd86-4b50-8598-72f59f04d193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:04.892491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-d9b9a391-dd86-4b50-8598-72f59f04d193 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:04.893696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d9b9a391-dd86-4b50-8598-72f59f04d193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:04.901428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:05.049742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53738e79-ce43-4913-91ab-8f6a021e99ab None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:05.064342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53738e79-ce43-4913-91ab-8f6a021e99ab None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:05.069856 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba19bd-b4a6-4933-9467-80b1ac9cb20f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:05.070494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba19bd-b4a6-4933-9467-80b1ac9cb20f service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:05.070776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba19bd-b4a6-4933-9467-80b1ac9cb20f service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:05.071052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba19bd-b4a6-4933-9467-80b1ac9cb20f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:05.080916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba19bd-b4a6-4933-9467-80b1ac9cb20f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:05.082495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba19bd-b4a6-4933-9467-80b1ac9cb20f service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:05.091919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 761/1518] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:08.215891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fac53-6af0-454a-9177-0334b9571e53 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:08.216477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fac53-6af0-454a-9177-0334b9571e53 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:08.216477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fac53-6af0-454a-9177-0334b9571e53 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:08.217882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 758/1519] 10.222.0.23 () {54 vars in 960 bytes} [Mon May 23 15:54:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:54:08.222993 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b463351-ad25-464c-b91a-92f8925a136d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:08.223451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b463351-ad25-464c-b91a-92f8925a136d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:08.223564 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b463351-ad25-464c-b91a-92f8925a136d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:08.242048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-4b463351-ad25-464c-b91a-92f8925a136d None None] MFA Rules not processed for user `312ed2b555424c6a96b3d972ffea5ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:08.243256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4b463351-ad25-464c-b91a-92f8925a136d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:08.251878 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 762/1520] 10.222.0.23 () {58 vars in 1057 bytes} [Mon May 23 15:54:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:08.259400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fac53-6af0-454a-9177-0334b9571e53 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:08.273487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fac53-6af0-454a-9177-0334b9571e53 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:08.279056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534df1-7e10-4d01-9ae3-a44894337b4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:08.279696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534df1-7e10-4d01-9ae3-a44894337b4d service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:08.279979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534df1-7e10-4d01-9ae3-a44894337b4d service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:08.280264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534df1-7e10-4d01-9ae3-a44894337b4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:08.289065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0534df1-7e10-4d01-9ae3-a44894337b4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:08.290645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0534df1-7e10-4d01-9ae3-a44894337b4d service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:08.301091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 759/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:54:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:16.336335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07946dae-37e8-4fdd-b658-1e0e6b64b8e0 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:16.336865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07946dae-37e8-4fdd-b658-1e0e6b64b8e0 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:16.336977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07946dae-37e8-4fdd-b658-1e0e6b64b8e0 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:16.338327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:54:16.343613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c50f6-1fec-4083-9b04-6253592b4c47 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:16.343994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c50f6-1fec-4083-9b04-6253592b4c47 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:16.344181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c50f6-1fec-4083-9b04-6253592b4c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:16.362450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-714c50f6-1fec-4083-9b04-6253592b4c47 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:16.363694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-714c50f6-1fec-4083-9b04-6253592b4c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:16.371434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:16.524212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07946dae-37e8-4fdd-b658-1e0e6b64b8e0 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:16.544294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07946dae-37e8-4fdd-b658-1e0e6b64b8e0 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:16.544708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e101a-19f0-44cd-8de8-5f0e4656d2c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:16.545348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e101a-19f0-44cd-8de8-5f0e4656d2c4 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:16.545630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e101a-19f0-44cd-8de8-5f0e4656d2c4 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:16.545903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e101a-19f0-44cd-8de8-5f0e4656d2c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:16.553840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e101a-19f0-44cd-8de8-5f0e4656d2c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:16.555399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e101a-19f0-44cd-8de8-5f0e4656d2c4 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:16.564899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 764/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:17.426321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2fd67-b8b6-44e8-b25c-b1758e835dad None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:17.426891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2fd67-b8b6-44e8-b25c-b1758e835dad None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:17.426891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2fd67-b8b6-44e8-b25c-b1758e835dad None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:17.428289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 15:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:54:17.433721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e4b45-a0c2-430a-9445-c7a20dde213c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:17.434134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e4b45-a0c2-430a-9445-c7a20dde213c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:17.434323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e4b45-a0c2-430a-9445-c7a20dde213c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:17.452446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-cb8e4b45-a0c2-430a-9445-c7a20dde213c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:17.453640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-cb8e4b45-a0c2-430a-9445-c7a20dde213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:17.461338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 15:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:17.612417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2fd67-b8b6-44e8-b25c-b1758e835dad None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:17.626995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2fd67-b8b6-44e8-b25c-b1758e835dad None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:17.632268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3958b62-e7c3-437d-9578-f7fc9f54c6d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:17.632920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3958b62-e7c3-437d-9578-f7fc9f54c6d9 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:17.633203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3958b62-e7c3-437d-9578-f7fc9f54c6d9 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:17.633475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3958b62-e7c3-437d-9578-f7fc9f54c6d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:17.641413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3958b62-e7c3-437d-9578-f7fc9f54c6d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:17.643234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3958b62-e7c3-437d-9578-f7fc9f54c6d9 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:17.652352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 762/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:21.924134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9390b9-8339-4b53-a5de-f6a84ca4af09 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:21.924765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9390b9-8339-4b53-a5de-f6a84ca4af09 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:21.924765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9390b9-8339-4b53-a5de-f6a84ca4af09 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:21.926102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:54:21.931207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2afea1-565d-443c-b7fa-d88d8ebb4b87 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:21.931659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2afea1-565d-443c-b7fa-d88d8ebb4b87 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:21.931772 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2afea1-565d-443c-b7fa-d88d8ebb4b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:21.952375 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-0b2afea1-565d-443c-b7fa-d88d8ebb4b87 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:21.953602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0b2afea1-565d-443c-b7fa-d88d8ebb4b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:21.961317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:21.997475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9390b9-8339-4b53-a5de-f6a84ca4af09 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:21.999150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6f9390b9-8339-4b53-a5de-f6a84ca4af09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:22.057070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9390b9-8339-4b53-a5de-f6a84ca4af09 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:22.061436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72e517-6491-4fd1-8c7d-13296c06c61e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:22.062079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72e517-6491-4fd1-8c7d-13296c06c61e service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:22.062384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72e517-6491-4fd1-8c7d-13296c06c61e service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:22.062641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72e517-6491-4fd1-8c7d-13296c06c61e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:22.071725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72e517-6491-4fd1-8c7d-13296c06c61e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:22.073243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72e517-6491-4fd1-8c7d-13296c06c61e service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:22.082632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 767/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:24.271034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50857f-86d2-4c8b-95d8-0a9c313f97a5 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:24.271575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50857f-86d2-4c8b-95d8-0a9c313f97a5 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:24.271575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50857f-86d2-4c8b-95d8-0a9c313f97a5 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:24.273018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:54:24.278869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b7648-f830-4ed9-bba2-668e90f1ec23 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:24.279368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b7648-f830-4ed9-bba2-668e90f1ec23 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:24.279472 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b7648-f830-4ed9-bba2-668e90f1ec23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:24.297797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-5d4b7648-f830-4ed9-bba2-668e90f1ec23 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:24.299041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5d4b7648-f830-4ed9-bba2-668e90f1ec23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:24.306975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:24.343091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50857f-86d2-4c8b-95d8-0a9c313f97a5 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:24.353733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50857f-86d2-4c8b-95d8-0a9c313f97a5 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:24.357216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00351ab-31ba-41cd-8a5b-dc0ae0ff5433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:24.357848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00351ab-31ba-41cd-8a5b-dc0ae0ff5433 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:24.358123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00351ab-31ba-41cd-8a5b-dc0ae0ff5433 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:24.358395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00351ab-31ba-41cd-8a5b-dc0ae0ff5433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:24.368518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00351ab-31ba-41cd-8a5b-dc0ae0ff5433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:24.371263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00351ab-31ba-41cd-8a5b-dc0ae0ff5433 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:24.379638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 765/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:54:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:54:28.133240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32cbfa-fa13-49e9-800a-e84dacbd6ae9 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:28.133810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32cbfa-fa13-49e9-800a-e84dacbd6ae9 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:28.133810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32cbfa-fa13-49e9-800a-e84dacbd6ae9 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:28.135148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:54:28.140530 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceedb5a-359b-4b6c-80d4-f7b01bc07c29 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:28.140979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceedb5a-359b-4b6c-80d4-f7b01bc07c29 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:28.141117 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceedb5a-359b-4b6c-80d4-f7b01bc07c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:28.159017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-3ceedb5a-359b-4b6c-80d4-f7b01bc07c29 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:28.160210 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3ceedb5a-359b-4b6c-80d4-f7b01bc07c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:28.168464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:28.436476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32cbfa-fa13-49e9-800a-e84dacbd6ae9 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:28.438152 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6b32cbfa-fa13-49e9-800a-e84dacbd6ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:28.495450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32cbfa-fa13-49e9-800a-e84dacbd6ae9 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:28.499032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78648467-cd49-43c0-826b-fc184b96f3d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:28.499678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648467-cd49-43c0-826b-fc184b96f3d5 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:28.499966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648467-cd49-43c0-826b-fc184b96f3d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:28.500246 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78648467-cd49-43c0-826b-fc184b96f3d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:28.509180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78648467-cd49-43c0-826b-fc184b96f3d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:28.510613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78648467-cd49-43c0-826b-fc184b96f3d5 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:28.515572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:28.520645 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 770/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:30.737547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a198-6026-47f3-8da5-238ea317ed05 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:30.738065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a198-6026-47f3-8da5-238ea317ed05 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:30.738065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a198-6026-47f3-8da5-238ea317ed05 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:30.739415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 15:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:54:30.744597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b2b8-ea35-486b-b41d-08230dea7116 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:30.745034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b2b8-ea35-486b-b41d-08230dea7116 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:30.745163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b2b8-ea35-486b-b41d-08230dea7116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:30.765245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-bd55b2b8-ea35-486b-b41d-08230dea7116 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:54:30.766437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bd55b2b8-ea35-486b-b41d-08230dea7116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:30.774350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 15:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:54:31.042567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a198-6026-47f3-8da5-238ea317ed05 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:54:31.044760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3bc1a198-6026-47f3-8da5-238ea317ed05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:54:31.057583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a198-6026-47f3-8da5-238ea317ed05 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:54:31.061559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d44622-09e4-4e87-80c4-b50007dfe7bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:54:31.062188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d44622-09e4-4e87-80c4-b50007dfe7bf service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:54:31.062468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d44622-09e4-4e87-80c4-b50007dfe7bf service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:54:31.062741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d44622-09e4-4e87-80c4-b50007dfe7bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:54:31.071374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d44622-09e4-4e87-80c4-b50007dfe7bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:54:31.072829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d44622-09e4-4e87-80c4-b50007dfe7bf service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:54:31.081906 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 768/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:54:40.759469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:54:41.077969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:00.843305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da97b7-e27a-4587-8a23-5aad4e2f31d0 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:00.848463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da97b7-e27a-4587-8a23-5aad4e2f31d0 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:00.848463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da97b7-e27a-4587-8a23-5aad4e2f31d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:00.885487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-36da97b7-e27a-4587-8a23-5aad4e2f31d0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:00.887118 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-36da97b7-e27a-4587-8a23-5aad4e2f31d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:00.897653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 772/1540] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:00.907427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3921d36-600a-4365-a1f6-0730d69384dc None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:00.907994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3921d36-600a-4365-a1f6-0730d69384dc None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:00.907994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3921d36-600a-4365-a1f6-0730d69384dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:00.930874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005d8ac-1bf1-47df-b646-5e307301e280 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:00.931558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005d8ac-1bf1-47df-b646-5e307301e280 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:00.932059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005d8ac-1bf1-47df-b646-5e307301e280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:00.941383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e3921d36-600a-4365-a1f6-0730d69384dc None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:00.945801 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e3921d36-600a-4365-a1f6-0730d69384dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:00.958434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 769/1541] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:00.960859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2005d8ac-1bf1-47df-b646-5e307301e280 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:00.962302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2005d8ac-1bf1-47df-b646-5e307301e280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:00.970173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3921d36-600a-4365-a1f6-0730d69384dc None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:00.978917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 773/1542] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:00.989155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2c5cc-4398-4c05-a088-52fa39d49aec None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:00.989488 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3921d36-600a-4365-a1f6-0730d69384dc None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:00.989993 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2c5cc-4398-4c05-a088-52fa39d49aec None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:00.990505 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2c5cc-4398-4c05-a088-52fa39d49aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.001472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9145d61-ac6b-40d2-af86-559109fd00f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.002345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9145d61-ac6b-40d2-af86-559109fd00f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.002625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9145d61-ac6b-40d2-af86-559109fd00f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.003837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9145d61-ac6b-40d2-af86-559109fd00f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.003837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9145d61-ac6b-40d2-af86-559109fd00f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:01.004172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9145d61-ac6b-40d2-af86-559109fd00f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.005224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: WARNING py.warnings [None req-a9145d61-ac6b-40d2-af86-559109fd00f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 15:56:01.005224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: warnings.warn(msg) May 23 15:56:01.005224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: May 23 15:56:01.006420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9145d61-ac6b-40d2-af86-559109fd00f3 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.021896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-47e2c5cc-4398-4c05-a088-52fa39d49aec None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.025220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-47e2c5cc-4398-4c05-a088-52fa39d49aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.033126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 770/1543] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:56:01.035891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 774/1544] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:01.043642 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9145d61-ac6b-40d2-af86-559109fd00f3 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.043828 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2c5cc-4398-4c05-a088-52fa39d49aec None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.066856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9145d61-ac6b-40d2-af86-559109fd00f3 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.067672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2c5cc-4398-4c05-a088-52fa39d49aec None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.076090 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8ec6d-de84-4023-8a06-c3bc161d23f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.081009 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a3958-f7c5-4c29-8f89-eb87f187c7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.082227 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8ec6d-de84-4023-8a06-c3bc161d23f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.082517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8ec6d-de84-4023-8a06-c3bc161d23f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.082797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8ec6d-de84-4023-8a06-c3bc161d23f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.083535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8ec6d-de84-4023-8a06-c3bc161d23f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:01.083848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8ec6d-de84-4023-8a06-c3bc161d23f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.085146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a3958-f7c5-4c29-8f89-eb87f187c7bc admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.085441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a3958-f7c5-4c29-8f89-eb87f187c7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.085721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a3958-f7c5-4c29-8f89-eb87f187c7bc admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.086423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a3958-f7c5-4c29-8f89-eb87f187c7bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.087445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WARNING py.warnings [None req-596a3958-f7c5-4c29-8f89-eb87f187c7bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:56:01.087445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: warnings.warn(msg) May 23 15:56:01.087445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: May 23 15:56:01.088673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a3958-f7c5-4c29-8f89-eb87f187c7bc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.090440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8ec6d-de84-4023-8a06-c3bc161d23f3 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.120205 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 771/1545] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:01.127885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8ec6d-de84-4023-8a06-c3bc161d23f3 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.141936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8ec6d-de84-4023-8a06-c3bc161d23f3 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.148755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea686749-fca7-492c-a63b-602527ef5a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.148755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea686749-fca7-492c-a63b-602527ef5a93 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.148755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea686749-fca7-492c-a63b-602527ef5a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.148755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea686749-fca7-492c-a63b-602527ef5a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.149378 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea686749-fca7-492c-a63b-602527ef5a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.151221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea686749-fca7-492c-a63b-602527ef5a93 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.249321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 775/1546] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/projects => generated 371 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:01.254878 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a3958-f7c5-4c29-8f89-eb87f187c7bc admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.289161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 772/1547] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:56:01.295190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea686749-fca7-492c-a63b-602527ef5a93 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.417859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea686749-fca7-492c-a63b-602527ef5a93 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.421685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a3958-f7c5-4c29-8f89-eb87f187c7bc admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.431533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0ba2e-5eaa-4fdc-9fc5-93c578360680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.431768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41141e2b-589f-4413-bbcb-097cee289795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.432580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41141e2b-589f-4413-bbcb-097cee289795 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.432873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41141e2b-589f-4413-bbcb-097cee289795 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.433154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41141e2b-589f-4413-bbcb-097cee289795 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.433358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0ba2e-5eaa-4fdc-9fc5-93c578360680 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.433811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0ba2e-5eaa-4fdc-9fc5-93c578360680 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.433927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41141e2b-589f-4413-bbcb-097cee289795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.434372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0ba2e-5eaa-4fdc-9fc5-93c578360680 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.435229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0ba2e-5eaa-4fdc-9fc5-93c578360680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.435801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41141e2b-589f-4413-bbcb-097cee289795 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.436664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: WARNING py.warnings [None req-bbf0ba2e-5eaa-4fdc-9fc5-93c578360680 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:56:01.436664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: warnings.warn(msg) May 23 15:56:01.436664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: May 23 15:56:01.438406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0ba2e-5eaa-4fdc-9fc5-93c578360680 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.564043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 773/1548] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/users => generated 294 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:01.565699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217773d-bc93-4682-a2f6-6a92ec5f6a1b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.566350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217773d-bc93-4682-a2f6-6a92ec5f6a1b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.566789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217773d-bc93-4682-a2f6-6a92ec5f6a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.586219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 776/1549] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/users => generated 286 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:56:01.589418 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41141e2b-589f-4413-bbcb-097cee289795 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.592517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e217773d-bc93-4682-a2f6-6a92ec5f6a1b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.593917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e217773d-bc93-4682-a2f6-6a92ec5f6a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.599642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41141e2b-589f-4413-bbcb-097cee289795 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.605726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc787e1c-ebf6-4245-92d9-65c9bd01f71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.606566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc787e1c-ebf6-4245-92d9-65c9bd01f71e admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.606861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc787e1c-ebf6-4245-92d9-65c9bd01f71e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.607146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc787e1c-ebf6-4245-92d9-65c9bd01f71e admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.607988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc787e1c-ebf6-4245-92d9-65c9bd01f71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.609554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 774/1550] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:01.610399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc787e1c-ebf6-4245-92d9-65c9bd01f71e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.616545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0468d8f-3769-4648-bc00-ea16025c0cc3 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.618090 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0468d8f-3769-4648-bc00-ea16025c0cc3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.618090 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0468d8f-3769-4648-bc00-ea16025c0cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.625334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 777/1551] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:01.625586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62c1cc-dc67-45f1-a305-d00efca5838e None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.625895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62c1cc-dc67-45f1-a305-d00efca5838e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.626185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62c1cc-dc67-45f1-a305-d00efca5838e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.647885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-6e62c1cc-dc67-45f1-a305-d00efca5838e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.649180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6e62c1cc-dc67-45f1-a305-d00efca5838e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.653493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-d0468d8f-3769-4648-bc00-ea16025c0cc3 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.655199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d0468d8f-3769-4648-bc00-ea16025c0cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.664074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 778/1552] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:01.666974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62c1cc-dc67-45f1-a305-d00efca5838e None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.674378 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 775/1553] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:01.678686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878d68e-a003-49e5-9aee-0fee1e097730 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.678977 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878d68e-a003-49e5-9aee-0fee1e097730 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.679258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878d68e-a003-49e5-9aee-0fee1e097730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.691609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62c1cc-dc67-45f1-a305-d00efca5838e None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.704504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-d878d68e-a003-49e5-9aee-0fee1e097730 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.705927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d878d68e-a003-49e5-9aee-0fee1e097730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.706635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe3252-ef53-4493-b51b-38926f3685f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.707414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe3252-ef53-4493-b51b-38926f3685f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.707757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe3252-ef53-4493-b51b-38926f3685f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.708100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe3252-ef53-4493-b51b-38926f3685f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.708936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe3252-ef53-4493-b51b-38926f3685f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.710604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe3252-ef53-4493-b51b-38926f3685f8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.720581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 776/1554] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:01.721481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878d68e-a003-49e5-9aee-0fee1e097730 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.731335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 779/1555] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:01.731575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878d68e-a003-49e5-9aee-0fee1e097730 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.732758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e571f27-28ab-44ac-b5e8-e458d170d252 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.733106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e571f27-28ab-44ac-b5e8-e458d170d252 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.733438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e571f27-28ab-44ac-b5e8-e458d170d252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.736629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235e019-e5fd-438c-8fff-d408550fa392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.737296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235e019-e5fd-438c-8fff-d408550fa392 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.737582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235e019-e5fd-438c-8fff-d408550fa392 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.737861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235e019-e5fd-438c-8fff-d408550fa392 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.738709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235e019-e5fd-438c-8fff-d408550fa392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.740300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235e019-e5fd-438c-8fff-d408550fa392 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.753560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 777/1556] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:01.753745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-1e571f27-28ab-44ac-b5e8-e458d170d252 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.753846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef85e3-7717-4aa5-a87a-a813138ff8be None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.753846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef85e3-7717-4aa5-a87a-a813138ff8be None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.753846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef85e3-7717-4aa5-a87a-a813138ff8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.754083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1e571f27-28ab-44ac-b5e8-e458d170d252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.761020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 780/1557] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:01.762220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1be18b-13a6-4fea-914a-b0f64dc454fb None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.762507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1be18b-13a6-4fea-914a-b0f64dc454fb None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.762783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1be18b-13a6-4fea-914a-b0f64dc454fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.770971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-48ef85e3-7717-4aa5-a87a-a813138ff8be None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.772161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-48ef85e3-7717-4aa5-a87a-a813138ff8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.779953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 778/1558] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:01.779953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d28a-902b-4c91-bbf3-6fa4f48ccc98 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.779953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d28a-902b-4c91-bbf3-6fa4f48ccc98 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.779953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67d28a-902b-4c91-bbf3-6fa4f48ccc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.785904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ca1be18b-13a6-4fea-914a-b0f64dc454fb None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.787162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ca1be18b-13a6-4fea-914a-b0f64dc454fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.795097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 781/1559] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:01.795949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1be18b-13a6-4fea-914a-b0f64dc454fb None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.800145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-de67d28a-902b-4c91-bbf3-6fa4f48ccc98 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.801368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-de67d28a-902b-4c91-bbf3-6fa4f48ccc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.805338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1be18b-13a6-4fea-914a-b0f64dc454fb None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.809262 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 779/1560] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:01.809773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0407cd-bf5c-433a-81ee-42434de443c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.810262 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2c6d-92d4-4a46-8d92-84d3dbd83037 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.810410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0407cd-bf5c-433a-81ee-42434de443c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.810549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2c6d-92d4-4a46-8d92-84d3dbd83037 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.810827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2c6d-92d4-4a46-8d92-84d3dbd83037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.811328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0407cd-bf5c-433a-81ee-42434de443c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.811618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0407cd-bf5c-433a-81ee-42434de443c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.812391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0407cd-bf5c-433a-81ee-42434de443c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:01.812681 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0407cd-bf5c-433a-81ee-42434de443c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.814314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0407cd-bf5c-433a-81ee-42434de443c8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.826335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 782/1561] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:01.827008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0407cd-bf5c-433a-81ee-42434de443c8 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.830214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-3dcd2c6d-92d4-4a46-8d92-84d3dbd83037 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.831400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3dcd2c6d-92d4-4a46-8d92-84d3dbd83037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.839651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 780/1562] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:01.840671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd2c6d-92d4-4a46-8d92-84d3dbd83037 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.844640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0407cd-bf5c-433a-81ee-42434de443c8 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.850773 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd2c6d-92d4-4a46-8d92-84d3dbd83037 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.855364 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4965958-9789-4150-82e6-48bd4d033d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.855574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ab417-7dda-432f-b774-6f523705bedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.856243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ab417-7dda-432f-b774-6f523705bedd admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.856538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4965958-9789-4150-82e6-48bd4d033d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.856649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ab417-7dda-432f-b774-6f523705bedd admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.856737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4965958-9789-4150-82e6-48bd4d033d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.856858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ab417-7dda-432f-b774-6f523705bedd admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.857006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4965958-9789-4150-82e6-48bd4d033d5d admin admin] PATH_INFO: `/v3/projects/b916e4deb43a43e7a8379aa285d3bdd6/users/0483214f1ab6452e92f627dffbbd5d44/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.857587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ab417-7dda-432f-b774-6f523705bedd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.859443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ab417-7dda-432f-b774-6f523705bedd admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.874640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 783/1563] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:01.874837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4965958-9789-4150-82e6-48bd4d033d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b916e4deb43a43e7a8379aa285d3bdd6, user_id=0483214f1ab6452e92f627dffbbd5d44, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.874950 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ab417-7dda-432f-b774-6f523705bedd admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.875030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4965958-9789-4150-82e6-48bd4d033d5d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.882200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ab417-7dda-432f-b774-6f523705bedd admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.886646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40749c99-d51d-4cea-b45f-73637d7baf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.887276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40749c99-d51d-4cea-b45f-73637d7baf53 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.887558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40749c99-d51d-4cea-b45f-73637d7baf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.887832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40749c99-d51d-4cea-b45f-73637d7baf53 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.888568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40749c99-d51d-4cea-b45f-73637d7baf53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:01.888871 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40749c99-d51d-4cea-b45f-73637d7baf53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.890507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40749c99-d51d-4cea-b45f-73637d7baf53 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.902812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 784/1564] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:01.903982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c43f39-f7ad-4a20-9cbf-d9d3e0cd9bca None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.904309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c43f39-f7ad-4a20-9cbf-d9d3e0cd9bca None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.904620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c43f39-f7ad-4a20-9cbf-d9d3e0cd9bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.922502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a2c43f39-f7ad-4a20-9cbf-d9d3e0cd9bca None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.923678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a2c43f39-f7ad-4a20-9cbf-d9d3e0cd9bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.931460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 785/1565] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:01.932203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c43f39-f7ad-4a20-9cbf-d9d3e0cd9bca None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.941147 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c43f39-f7ad-4a20-9cbf-d9d3e0cd9bca None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:01.945517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727618cd-54f8-4bec-b120-777085cfe167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:01.946136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727618cd-54f8-4bec-b120-777085cfe167 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.946414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727618cd-54f8-4bec-b120-777085cfe167 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.946686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727618cd-54f8-4bec-b120-777085cfe167 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.947384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727618cd-54f8-4bec-b120-777085cfe167 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:01.947685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727618cd-54f8-4bec-b120-777085cfe167 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:01.949282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727618cd-54f8-4bec-b120-777085cfe167 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:01.961419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 786/1566] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:01.962549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c96e-eb10-4fbe-b74a-3d6fa61f42aa None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:01.962836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c96e-eb10-4fbe-b74a-3d6fa61f42aa None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:01.963117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c96e-eb10-4fbe-b74a-3d6fa61f42aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:01.979812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 781/1567] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:01 2022] PUT /identity/v3/projects/b916e4deb43a43e7a8379aa285d3bdd6/users/0483214f1ab6452e92f627dffbbd5d44/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:01.982610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2c02c96e-eb10-4fbe-b74a-3d6fa61f42aa None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:01.982746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4965958-9789-4150-82e6-48bd4d033d5d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:01.982960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2c02c96e-eb10-4fbe-b74a-3d6fa61f42aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:01.987764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:02.004028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:02.045288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 787/1568] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:02.046189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4965958-9789-4150-82e6-48bd4d033d5d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.046402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73536bfb-5b65-4cd7-9886-2418039fae30 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.046699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73536bfb-5b65-4cd7-9886-2418039fae30 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.046980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73536bfb-5b65-4cd7-9886-2418039fae30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.050967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401711b6-79fe-4eda-bfac-02dc92443c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.051671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401711b6-79fe-4eda-bfac-02dc92443c50 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.051971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401711b6-79fe-4eda-bfac-02dc92443c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.052232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401711b6-79fe-4eda-bfac-02dc92443c50 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.052954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401711b6-79fe-4eda-bfac-02dc92443c50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.055145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401711b6-79fe-4eda-bfac-02dc92443c50 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.065091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-73536bfb-5b65-4cd7-9886-2418039fae30 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:02.066295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-73536bfb-5b65-4cd7-9886-2418039fae30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:02.074215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 788/1569] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:02.074976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73536bfb-5b65-4cd7-9886-2418039fae30 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.085193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73536bfb-5b65-4cd7-9886-2418039fae30 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.089588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84dacc-13d5-4c6e-b24f-7cb0f07bdc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.090474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84dacc-13d5-4c6e-b24f-7cb0f07bdc7a admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.090843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84dacc-13d5-4c6e-b24f-7cb0f07bdc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.091124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84dacc-13d5-4c6e-b24f-7cb0f07bdc7a admin admin] PATH_INFO: `/v3/projects/cb224c26e34943d996918e3762c1e965/users/f32d1bd8cad44434a6bd04bf04170d28/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.106077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84dacc-13d5-4c6e-b24f-7cb0f07bdc7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb224c26e34943d996918e3762c1e965, user_id=f32d1bd8cad44434a6bd04bf04170d28, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.108787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WARNING py.warnings [None req-cc84dacc-13d5-4c6e-b24f-7cb0f07bdc7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:56:02.108787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: warnings.warn(msg) May 23 15:56:02.108787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: May 23 15:56:02.109129 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84dacc-13d5-4c6e-b24f-7cb0f07bdc7a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.174090 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 782/1570] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:01 2022] POST /identity/v3/projects => generated 393 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:02.174946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401711b6-79fe-4eda-bfac-02dc92443c50 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.195205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 789/1571] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:02 2022] PUT /identity/v3/projects/cb224c26e34943d996918e3762c1e965/users/f32d1bd8cad44434a6bd04bf04170d28/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:02.196211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84dacc-13d5-4c6e-b24f-7cb0f07bdc7a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.246158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401711b6-79fe-4eda-bfac-02dc92443c50 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.259278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84dacc-13d5-4c6e-b24f-7cb0f07bdc7a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.265025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc889726-55fb-4890-8dd2-1e1107ac6a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.266190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc889726-55fb-4890-8dd2-1e1107ac6a9d admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.266476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc889726-55fb-4890-8dd2-1e1107ac6a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.266756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc889726-55fb-4890-8dd2-1e1107ac6a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.267614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc889726-55fb-4890-8dd2-1e1107ac6a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:02.267928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc889726-55fb-4890-8dd2-1e1107ac6a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.269970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc889726-55fb-4890-8dd2-1e1107ac6a9d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.283258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 790/1572] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:02.287172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc889726-55fb-4890-8dd2-1e1107ac6a9d admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.294912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc889726-55fb-4890-8dd2-1e1107ac6a9d admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.300111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fdba6-b794-475e-9693-8b6f38239606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.300796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fdba6-b794-475e-9693-8b6f38239606 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.301087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fdba6-b794-475e-9693-8b6f38239606 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.301368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fdba6-b794-475e-9693-8b6f38239606 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.302055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fdba6-b794-475e-9693-8b6f38239606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.303990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fdba6-b794-475e-9693-8b6f38239606 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.308082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1de0e-839f-4155-8371-a05d09947456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.308616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1de0e-839f-4155-8371-a05d09947456 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.308933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1de0e-839f-4155-8371-a05d09947456 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.309212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1de0e-839f-4155-8371-a05d09947456 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.309905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1de0e-839f-4155-8371-a05d09947456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.311843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1de0e-839f-4155-8371-a05d09947456 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.406032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 791/1573] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:02.407809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09899f-7722-4fc0-bbf7-130f74709e43 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.408098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09899f-7722-4fc0-bbf7-130f74709e43 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.408393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09899f-7722-4fc0-bbf7-130f74709e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.417634 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 783/1574] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/projects => generated 379 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:02.418472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1de0e-839f-4155-8371-a05d09947456 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.435666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0b09899f-7722-4fc0-bbf7-130f74709e43 None None] MFA Rules not processed for user `0483214f1ab6452e92f627dffbbd5d44`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:02.436821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0b09899f-7722-4fc0-bbf7-130f74709e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:02.463966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:02.477564 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:02.484813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1de0e-839f-4155-8371-a05d09947456 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.489460 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c63e8-8b9f-45cb-b229-8ca08d529e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.490131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c63e8-8b9f-45cb-b229-8ca08d529e1f admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.490426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c63e8-8b9f-45cb-b229-8ca08d529e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.490703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c63e8-8b9f-45cb-b229-8ca08d529e1f admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.491438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c63e8-8b9f-45cb-b229-8ca08d529e1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:02.491769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c63e8-8b9f-45cb-b229-8ca08d529e1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.493380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c63e8-8b9f-45cb-b229-8ca08d529e1f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.500763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 792/1575] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:02.501877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09899f-7722-4fc0-bbf7-130f74709e43 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.508404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 784/1576] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:02.509520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c63e8-8b9f-45cb-b229-8ca08d529e1f admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.513095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09899f-7722-4fc0-bbf7-130f74709e43 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.518345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26eaf14-6fbb-4f75-ba38-a1b2aca2dae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.519059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eaf14-6fbb-4f75-ba38-a1b2aca2dae8 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.519356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eaf14-6fbb-4f75-ba38-a1b2aca2dae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.519635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eaf14-6fbb-4f75-ba38-a1b2aca2dae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.519735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c63e8-8b9f-45cb-b229-8ca08d529e1f admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.520384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26eaf14-6fbb-4f75-ba38-a1b2aca2dae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:02.521049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26eaf14-6fbb-4f75-ba38-a1b2aca2dae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.522690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26eaf14-6fbb-4f75-ba38-a1b2aca2dae8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.524846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d28ab-9c75-4f29-8257-3d0524c748b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.525513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d28ab-9c75-4f29-8257-3d0524c748b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.525915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d28ab-9c75-4f29-8257-3d0524c748b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.526200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d28ab-9c75-4f29-8257-3d0524c748b2 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.526894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d28ab-9c75-4f29-8257-3d0524c748b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.528867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d28ab-9c75-4f29-8257-3d0524c748b2 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.535648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 793/1577] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:02.536984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e48fd6-d214-43f3-8b92-e204968ac9f8 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.537280 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e48fd6-d214-43f3-8b92-e204968ac9f8 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.537565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e48fd6-d214-43f3-8b92-e204968ac9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.563102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a2e48fd6-d214-43f3-8b92-e204968ac9f8 None None] MFA Rules not processed for user `f32d1bd8cad44434a6bd04bf04170d28`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:02.564351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a2e48fd6-d214-43f3-8b92-e204968ac9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:02.601292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 785/1578] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:02.602176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d28ab-9c75-4f29-8257-3d0524c748b2 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.611978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d28ab-9c75-4f29-8257-3d0524c748b2 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.616625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7f482-60b1-435b-b582-17c1c1bd531a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.617271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7f482-60b1-435b-b582-17c1c1bd531a admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.617554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7f482-60b1-435b-b582-17c1c1bd531a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.617832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7f482-60b1-435b-b582-17c1c1bd531a admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.618515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7f482-60b1-435b-b582-17c1c1bd531a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.620431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7f482-60b1-435b-b582-17c1c1bd531a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.629008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 794/1579] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:02.629008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e48fd6-d214-43f3-8b92-e204968ac9f8 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.639943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e48fd6-d214-43f3-8b92-e204968ac9f8 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.645038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af210c7-7d80-4b7c-8b66-8f8e358a21fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.645983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af210c7-7d80-4b7c-8b66-8f8e358a21fc admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.646270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af210c7-7d80-4b7c-8b66-8f8e358a21fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.646548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af210c7-7d80-4b7c-8b66-8f8e358a21fc admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.647236 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af210c7-7d80-4b7c-8b66-8f8e358a21fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.649302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af210c7-7d80-4b7c-8b66-8f8e358a21fc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.697453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 786/1580] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:02.698202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7f482-60b1-435b-b582-17c1c1bd531a admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.722803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 795/1581] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:02.722803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af210c7-7d80-4b7c-8b66-8f8e358a21fc admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.732488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af210c7-7d80-4b7c-8b66-8f8e358a21fc admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.759443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7f482-60b1-435b-b582-17c1c1bd531a admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.764337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae35aa8-6b42-41c6-b516-23ad3817c006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.765155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae35aa8-6b42-41c6-b516-23ad3817c006 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.765445 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae35aa8-6b42-41c6-b516-23ad3817c006 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.765723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae35aa8-6b42-41c6-b516-23ad3817c006 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.766949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae35aa8-6b42-41c6-b516-23ad3817c006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.768947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae35aa8-6b42-41c6-b516-23ad3817c006 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.790511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95776bde-2e6d-493c-a781-838edd713efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.791179 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95776bde-2e6d-493c-a781-838edd713efc admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.791494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95776bde-2e6d-493c-a781-838edd713efc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.791752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95776bde-2e6d-493c-a781-838edd713efc admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.792487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95776bde-2e6d-493c-a781-838edd713efc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.794417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95776bde-2e6d-493c-a781-838edd713efc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.873605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 787/1582] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:02.874472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae35aa8-6b42-41c6-b516-23ad3817c006 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.884348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae35aa8-6b42-41c6-b516-23ad3817c006 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.887343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 796/1583] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:02.888142 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95776bde-2e6d-493c-a781-838edd713efc admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:02.890876 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383a7aa-8d62-4c90-bd55-b9ffa77549b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.892076 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383a7aa-8d62-4c90-bd55-b9ffa77549b3 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.892384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383a7aa-8d62-4c90-bd55-b9ffa77549b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.892665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383a7aa-8d62-4c90-bd55-b9ffa77549b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.893353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383a7aa-8d62-4c90-bd55-b9ffa77549b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.895417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383a7aa-8d62-4c90-bd55-b9ffa77549b3 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.916759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:02.950364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95776bde-2e6d-493c-a781-838edd713efc admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:02.955390 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a7bc7-34a2-43e3-8843-cd4dfbae796f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:02.956054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a7bc7-34a2-43e3-8843-cd4dfbae796f admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:02.956339 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a7bc7-34a2-43e3-8843-cd4dfbae796f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:02.956655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a7bc7-34a2-43e3-8843-cd4dfbae796f admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:02.957348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a7bc7-34a2-43e3-8843-cd4dfbae796f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:02.959253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a7bc7-34a2-43e3-8843-cd4dfbae796f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:02.984848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 788/1584] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:02.985680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383a7aa-8d62-4c90-bd55-b9ffa77549b3 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.047925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383a7aa-8d62-4c90-bd55-b9ffa77549b3 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.052766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0f077-aa95-4e7d-9a45-de610add12d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.053477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0f077-aa95-4e7d-9a45-de610add12d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.053762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0f077-aa95-4e7d-9a45-de610add12d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.054048 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0f077-aa95-4e7d-9a45-de610add12d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.054814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0f077-aa95-4e7d-9a45-de610add12d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.056470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0f077-aa95-4e7d-9a45-de610add12d9 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.059331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:03.063444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 797/1585] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:02 2022] POST /identity/v3/projects => generated 403 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:03.063444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a7bc7-34a2-43e3-8843-cd4dfbae796f admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.070564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 789/1586] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.070564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0f077-aa95-4e7d-9a45-de610add12d9 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.077926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0f077-aa95-4e7d-9a45-de610add12d9 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.128757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a7bc7-34a2-43e3-8843-cd4dfbae796f admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.133418 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12099b-f966-44da-b2a8-65329d1750d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.133924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1de881-d4f1-480e-bfca-5c050a9761b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.134070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12099b-f966-44da-b2a8-65329d1750d9 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.134350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12099b-f966-44da-b2a8-65329d1750d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.134599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1de881-d4f1-480e-bfca-5c050a9761b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.134725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12099b-f966-44da-b2a8-65329d1750d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.134888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1de881-d4f1-480e-bfca-5c050a9761b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.135166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1de881-d4f1-480e-bfca-5c050a9761b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.135304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12099b-f966-44da-b2a8-65329d1750d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.135856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1de881-d4f1-480e-bfca-5c050a9761b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.137243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12099b-f966-44da-b2a8-65329d1750d9 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.137803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1de881-d4f1-480e-bfca-5c050a9761b7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.222035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 790/1587] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:03 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:03.222781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1de881-d4f1-480e-bfca-5c050a9761b7 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.232158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1de881-d4f1-480e-bfca-5c050a9761b7 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.237373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196869de-c1a6-4cbd-81d1-94148d636c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.238029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196869de-c1a6-4cbd-81d1-94148d636c56 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.238297 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196869de-c1a6-4cbd-81d1-94148d636c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.238573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196869de-c1a6-4cbd-81d1-94148d636c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.239329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196869de-c1a6-4cbd-81d1-94148d636c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.240937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196869de-c1a6-4cbd-81d1-94148d636c56 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.245336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 798/1588] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:03 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:03.245336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12099b-f966-44da-b2a8-65329d1750d9 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.257732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 791/1589] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.257732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196869de-c1a6-4cbd-81d1-94148d636c56 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.265097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196869de-c1a6-4cbd-81d1-94148d636c56 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.309410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12099b-f966-44da-b2a8-65329d1750d9 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.314629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23e86b-f1d3-4ca5-ae4b-9274ba5118ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.315298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23e86b-f1d3-4ca5-ae4b-9274ba5118ca admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.315600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23e86b-f1d3-4ca5-ae4b-9274ba5118ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.315862 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23e86b-f1d3-4ca5-ae4b-9274ba5118ca admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.316625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23e86b-f1d3-4ca5-ae4b-9274ba5118ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.318218 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23e86b-f1d3-4ca5-ae4b-9274ba5118ca admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.326731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bac9e-27c1-466f-aa9c-e50b82e476cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.327511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bac9e-27c1-466f-aa9c-e50b82e476cb admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.327953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 799/1590] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.328089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bac9e-27c1-466f-aa9c-e50b82e476cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.328477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bac9e-27c1-466f-aa9c-e50b82e476cb admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.328850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23e86b-f1d3-4ca5-ae4b-9274ba5118ca admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.329242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bac9e-27c1-466f-aa9c-e50b82e476cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.331083 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bac9e-27c1-466f-aa9c-e50b82e476cb admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.338746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23e86b-f1d3-4ca5-ae4b-9274ba5118ca admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.343547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a02bcd-0d8c-41ab-92c7-bcbd4298ff75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.344200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a02bcd-0d8c-41ab-92c7-bcbd4298ff75 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.344534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a02bcd-0d8c-41ab-92c7-bcbd4298ff75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.344814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a02bcd-0d8c-41ab-92c7-bcbd4298ff75 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.345493 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a02bcd-0d8c-41ab-92c7-bcbd4298ff75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.347400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a02bcd-0d8c-41ab-92c7-bcbd4298ff75 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.399682 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 792/1591] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:03 2022] POST /identity/v3/users => generated 299 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:03.400757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bac9e-27c1-466f-aa9c-e50b82e476cb admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.410184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bac9e-27c1-466f-aa9c-e50b82e476cb admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.414805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26881e-671b-4662-884c-2bef267b1dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.415445 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26881e-671b-4662-884c-2bef267b1dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.415730 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26881e-671b-4662-884c-2bef267b1dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.416008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26881e-671b-4662-884c-2bef267b1dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.416723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26881e-671b-4662-884c-2bef267b1dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.418616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26881e-671b-4662-884c-2bef267b1dc3 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.428055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 800/1592] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:03 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:03.429030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a02bcd-0d8c-41ab-92c7-bcbd4298ff75 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.438999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a02bcd-0d8c-41ab-92c7-bcbd4298ff75 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.443656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebaae4-273a-402a-91c5-1d4e311f49d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.444300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebaae4-273a-402a-91c5-1d4e311f49d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.444632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebaae4-273a-402a-91c5-1d4e311f49d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.444914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebaae4-273a-402a-91c5-1d4e311f49d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.445641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebaae4-273a-402a-91c5-1d4e311f49d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.447209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebaae4-273a-402a-91c5-1d4e311f49d3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.457724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 801/1593] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.458490 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebaae4-273a-402a-91c5-1d4e311f49d3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.463260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:03.468262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebaae4-273a-402a-91c5-1d4e311f49d3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.472866 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a2c3c-610b-4dcf-8fb8-a8de1f665261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.473435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a2c3c-610b-4dcf-8fb8-a8de1f665261 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.473721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a2c3c-610b-4dcf-8fb8-a8de1f665261 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.474006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a2c3c-610b-4dcf-8fb8-a8de1f665261 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.474705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a2c3c-610b-4dcf-8fb8-a8de1f665261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.476243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a2c3c-610b-4dcf-8fb8-a8de1f665261 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.486373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 802/1594] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.487117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a2c3c-610b-4dcf-8fb8-a8de1f665261 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.496577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a2c3c-610b-4dcf-8fb8-a8de1f665261 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.501775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd7706-d512-474c-a007-d55c899a491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.502458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd7706-d512-474c-a007-d55c899a491e admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.505910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 793/1595] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:03 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:03.506048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd7706-d512-474c-a007-d55c899a491e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.506048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd7706-d512-474c-a007-d55c899a491e admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.506245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26881e-671b-4662-884c-2bef267b1dc3 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.506348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd7706-d512-474c-a007-d55c899a491e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.506619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd7706-d512-474c-a007-d55c899a491e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.516192 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26881e-671b-4662-884c-2bef267b1dc3 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.521379 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c151bbb-a9b2-4acf-b71a-aa39cd8611bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.522041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c151bbb-a9b2-4acf-b71a-aa39cd8611bb admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.522347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c151bbb-a9b2-4acf-b71a-aa39cd8611bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.522629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c151bbb-a9b2-4acf-b71a-aa39cd8611bb admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.523349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c151bbb-a9b2-4acf-b71a-aa39cd8611bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.524962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c151bbb-a9b2-4acf-b71a-aa39cd8611bb admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.536317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 794/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.537117 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c151bbb-a9b2-4acf-b71a-aa39cd8611bb admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.546206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c151bbb-a9b2-4acf-b71a-aa39cd8611bb admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.551433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4ff88-192b-457b-a1f0-5c88272e279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.552240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4ff88-192b-457b-a1f0-5c88272e279e admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.552644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4ff88-192b-457b-a1f0-5c88272e279e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.553009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4ff88-192b-457b-a1f0-5c88272e279e admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.553856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4ff88-192b-457b-a1f0-5c88272e279e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.555387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4ff88-192b-457b-a1f0-5c88272e279e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.565056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 795/1597] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.565824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4ff88-192b-457b-a1f0-5c88272e279e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.575454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4ff88-192b-457b-a1f0-5c88272e279e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.579881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137dccc-4422-4f34-9807-8730c314c472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.580533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137dccc-4422-4f34-9807-8730c314c472 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.580840 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137dccc-4422-4f34-9807-8730c314c472 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.581117 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137dccc-4422-4f34-9807-8730c314c472 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.581824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137dccc-4422-4f34-9807-8730c314c472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.583426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 803/1598] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:03 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:03.583426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd7706-d512-474c-a007-d55c899a491e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.583641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137dccc-4422-4f34-9807-8730c314c472 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.592338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd7706-d512-474c-a007-d55c899a491e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.594242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 796/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.595250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137dccc-4422-4f34-9807-8730c314c472 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.600691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5edc6-dedb-4a02-b8b1-b2690a192abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.601351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5edc6-dedb-4a02-b8b1-b2690a192abe admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.601643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5edc6-dedb-4a02-b8b1-b2690a192abe admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.601912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5edc6-dedb-4a02-b8b1-b2690a192abe admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.602728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5edc6-dedb-4a02-b8b1-b2690a192abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.604230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5edc6-dedb-4a02-b8b1-b2690a192abe admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.605161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137dccc-4422-4f34-9807-8730c314c472 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.609760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa7693-2076-418e-bde8-d34be9d46161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.611182 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa7693-2076-418e-bde8-d34be9d46161 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.611469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa7693-2076-418e-bde8-d34be9d46161 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.611746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa7693-2076-418e-bde8-d34be9d46161 admin admin] PATH_INFO: `/v3/projects/b795cdc9b51647bf96a3da010d94c9cd/users/5b52924339de4cc08fe35dbf2a5fdcd0/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.615737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 804/1600] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.616208 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5edc6-dedb-4a02-b8b1-b2690a192abe admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.625091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:03.626169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5edc6-dedb-4a02-b8b1-b2690a192abe admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.629249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa7693-2076-418e-bde8-d34be9d46161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b795cdc9b51647bf96a3da010d94c9cd, user_id=5b52924339de4cc08fe35dbf2a5fdcd0, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.630629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e71e14-9c0b-4594-9f21-be84abf543fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.631274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e71e14-9c0b-4594-9f21-be84abf543fa admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.631559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e71e14-9c0b-4594-9f21-be84abf543fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.631835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e71e14-9c0b-4594-9f21-be84abf543fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.632047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa7693-2076-418e-bde8-d34be9d46161 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.632603 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e71e14-9c0b-4594-9f21-be84abf543fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.634211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e71e14-9c0b-4594-9f21-be84abf543fa admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.644083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 805/1601] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.645927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e71e14-9c0b-4594-9f21-be84abf543fa admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.654213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e71e14-9c0b-4594-9f21-be84abf543fa admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.659228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcd407-1db8-46ef-b240-f935d7a30c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.659845 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcd407-1db8-46ef-b240-f935d7a30c4c admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.660128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcd407-1db8-46ef-b240-f935d7a30c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.660436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcd407-1db8-46ef-b240-f935d7a30c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.661138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcd407-1db8-46ef-b240-f935d7a30c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.662690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcd407-1db8-46ef-b240-f935d7a30c4c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.672208 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 806/1602] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.673029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcd407-1db8-46ef-b240-f935d7a30c4c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.682521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcd407-1db8-46ef-b240-f935d7a30c4c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.687079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddb44d-4daa-4ca1-8d55-a08b2a412639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.687983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddb44d-4daa-4ca1-8d55-a08b2a412639 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.688265 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddb44d-4daa-4ca1-8d55-a08b2a412639 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.688590 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddb44d-4daa-4ca1-8d55-a08b2a412639 admin admin] PATH_INFO: `/v3/projects/453b7ba3cabe4e2b805865ea2c8945bc/users/b3f3a9a74bb543d494845c394fe0c55f/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.691631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 797/1603] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:03 2022] PUT /identity/v3/projects/b795cdc9b51647bf96a3da010d94c9cd/users/5b52924339de4cc08fe35dbf2a5fdcd0/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:03.692344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa7693-2076-418e-bde8-d34be9d46161 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.708186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddb44d-4daa-4ca1-8d55-a08b2a412639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=453b7ba3cabe4e2b805865ea2c8945bc, user_id=b3f3a9a74bb543d494845c394fe0c55f, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.710844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddb44d-4daa-4ca1-8d55-a08b2a412639 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.756119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa7693-2076-418e-bde8-d34be9d46161 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.760836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88664c23-1d8c-4930-9e33-79b22b101cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.761717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88664c23-1d8c-4930-9e33-79b22b101cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.762001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88664c23-1d8c-4930-9e33-79b22b101cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.762280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88664c23-1d8c-4930-9e33-79b22b101cdd admin admin] PATH_INFO: `/v3/projects/5acfb4c0798d48dda4afb4cf2ea06ed0/users/c2075a6ceede424087448411badff88b/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.771653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 807/1604] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:03 2022] PUT /identity/v3/projects/453b7ba3cabe4e2b805865ea2c8945bc/users/b3f3a9a74bb543d494845c394fe0c55f/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:03.772876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddb44d-4daa-4ca1-8d55-a08b2a412639 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.777113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88664c23-1d8c-4930-9e33-79b22b101cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5acfb4c0798d48dda4afb4cf2ea06ed0, user_id=c2075a6ceede424087448411badff88b, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.779918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88664c23-1d8c-4930-9e33-79b22b101cdd admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.834288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddb44d-4daa-4ca1-8d55-a08b2a412639 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.839085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8e7d5-ca2d-42ff-aaee-985683bc7d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.839829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8e7d5-ca2d-42ff-aaee-985683bc7d4f admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.840044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8e7d5-ca2d-42ff-aaee-985683bc7d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.840322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8e7d5-ca2d-42ff-aaee-985683bc7d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.841100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8e7d5-ca2d-42ff-aaee-985683bc7d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.842710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8e7d5-ca2d-42ff-aaee-985683bc7d4f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.845421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 798/1605] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:03 2022] PUT /identity/v3/projects/5acfb4c0798d48dda4afb4cf2ea06ed0/users/c2075a6ceede424087448411badff88b/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:03.846231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88664c23-1d8c-4930-9e33-79b22b101cdd admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.853794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 808/1606] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.861030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8e7d5-ca2d-42ff-aaee-985683bc7d4f admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.865769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8e7d5-ca2d-42ff-aaee-985683bc7d4f admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.915280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88664c23-1d8c-4930-9e33-79b22b101cdd admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.920281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0233b8-95ac-4b13-a8b2-5054c3a23f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.921328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0233b8-95ac-4b13-a8b2-5054c3a23f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.921749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0233b8-95ac-4b13-a8b2-5054c3a23f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.922044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0233b8-95ac-4b13-a8b2-5054c3a23f1c admin admin] PATH_INFO: `/v3/projects/523b990b083e468f82994360658a6c3b/users/ab93bf2cebb04d21b49e3bdce650513d/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.922352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bcfba-e7df-4d7a-9345-78bad2ef441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.923169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bcfba-e7df-4d7a-9345-78bad2ef441a admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.923451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bcfba-e7df-4d7a-9345-78bad2ef441a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.923728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bcfba-e7df-4d7a-9345-78bad2ef441a admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.924486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bcfba-e7df-4d7a-9345-78bad2ef441a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.926079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bcfba-e7df-4d7a-9345-78bad2ef441a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.935591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 809/1607] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:03.936352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bcfba-e7df-4d7a-9345-78bad2ef441a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:03.936644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0233b8-95ac-4b13-a8b2-5054c3a23f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=523b990b083e468f82994360658a6c3b, user_id=ab93bf2cebb04d21b49e3bdce650513d, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.939299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0233b8-95ac-4b13-a8b2-5054c3a23f1c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:03.947736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bcfba-e7df-4d7a-9345-78bad2ef441a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:03.952440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad27ead-2e01-4452-9a9e-00798fe883b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:03.953305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad27ead-2e01-4452-9a9e-00798fe883b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:03.953875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad27ead-2e01-4452-9a9e-00798fe883b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:03.953875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad27ead-2e01-4452-9a9e-00798fe883b6 admin admin] PATH_INFO: `/v3/projects/fce9058356d34a8e875b5e166b4ac0a9/users/3552d4b841ad47a2a89d3defa67e521b/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:03.957700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:03.969195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad27ead-2e01-4452-9a9e-00798fe883b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce9058356d34a8e875b5e166b4ac0a9, user_id=3552d4b841ad47a2a89d3defa67e521b, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:03.971903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad27ead-2e01-4452-9a9e-00798fe883b6 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.006882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 799/1608] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:03 2022] PUT /identity/v3/projects/523b990b083e468f82994360658a6c3b/users/ab93bf2cebb04d21b49e3bdce650513d/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:04.007725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0233b8-95ac-4b13-a8b2-5054c3a23f1c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.047050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 810/1609] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:03 2022] PUT /identity/v3/projects/fce9058356d34a8e875b5e166b4ac0a9/users/3552d4b841ad47a2a89d3defa67e521b/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:04.048400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d3c21-677b-4cdc-ad23-3b3c2cf0989f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.048690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d3c21-677b-4cdc-ad23-3b3c2cf0989f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.049039 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d3c21-677b-4cdc-ad23-3b3c2cf0989f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.067174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ae5d3c21-677b-4cdc-ad23-3b3c2cf0989f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:04.068390 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ae5d3c21-677b-4cdc-ad23-3b3c2cf0989f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:04.077412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:04.083227 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0233b8-95ac-4b13-a8b2-5054c3a23f1c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.128589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 811/1610] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:04.132107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d265c8-93a8-4902-86f4-1e0737f6e774 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.132107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d265c8-93a8-4902-86f4-1e0737f6e774 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.132107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d265c8-93a8-4902-86f4-1e0737f6e774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.143997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404a64c-8cbf-400a-a664-b69b460c933b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.144762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404a64c-8cbf-400a-a664-b69b460c933b admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.145056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404a64c-8cbf-400a-a664-b69b460c933b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.145334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404a64c-8cbf-400a-a664-b69b460c933b admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.146329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404a64c-8cbf-400a-a664-b69b460c933b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.147929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404a64c-8cbf-400a-a664-b69b460c933b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.158387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 800/1611] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:04.159818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-77d265c8-93a8-4902-86f4-1e0737f6e774 None None] MFA Rules not processed for user `b3f3a9a74bb543d494845c394fe0c55f`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:04.159966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a6e9c-27b2-4300-a399-b04cd9d88a75 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.160140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a6e9c-27b2-4300-a399-b04cd9d88a75 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.160931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a6e9c-27b2-4300-a399-b04cd9d88a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.161076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-77d265c8-93a8-4902-86f4-1e0737f6e774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:04.178578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-318a6e9c-27b2-4300-a399-b04cd9d88a75 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:04.179794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-318a6e9c-27b2-4300-a399-b04cd9d88a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:04.188118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 801/1612] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:04.189560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a6e9c-27b2-4300-a399-b04cd9d88a75 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.199642 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a6e9c-27b2-4300-a399-b04cd9d88a75 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.204794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a21819-5e7a-4bc0-bcc0-737935655af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.205677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a21819-5e7a-4bc0-bcc0-737935655af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.205961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a21819-5e7a-4bc0-bcc0-737935655af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.206236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a21819-5e7a-4bc0-bcc0-737935655af1 admin admin] PATH_INFO: `/v3/projects/8a024d1458954153b26bf70f811fa1b2/users/e9210de124314c3ba8dab95c6d3c95a3/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.221387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a21819-5e7a-4bc0-bcc0-737935655af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a024d1458954153b26bf70f811fa1b2, user_id=e9210de124314c3ba8dab95c6d3c95a3, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.224174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a21819-5e7a-4bc0-bcc0-737935655af1 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.228530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 812/1613] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:04.229414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d265c8-93a8-4902-86f4-1e0737f6e774 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.239425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d265c8-93a8-4902-86f4-1e0737f6e774 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.243940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c167f8-9718-4504-bf53-9a40f2a9edcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.245072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c167f8-9718-4504-bf53-9a40f2a9edcc admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.245190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c167f8-9718-4504-bf53-9a40f2a9edcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.245452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c167f8-9718-4504-bf53-9a40f2a9edcc admin admin] PATH_INFO: `/v3/projects/fc7d718373b94bd3b0c0ff86cd0975bd/users/83bb340d10e44bedad5fc9ddaf332552/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.261035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c167f8-9718-4504-bf53-9a40f2a9edcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7d718373b94bd3b0c0ff86cd0975bd, user_id=83bb340d10e44bedad5fc9ddaf332552, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.263634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c167f8-9718-4504-bf53-9a40f2a9edcc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.296464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 802/1614] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:04 2022] PUT /identity/v3/projects/8a024d1458954153b26bf70f811fa1b2/users/e9210de124314c3ba8dab95c6d3c95a3/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:04.297294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a21819-5e7a-4bc0-bcc0-737935655af1 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.357656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a21819-5e7a-4bc0-bcc0-737935655af1 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.362626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf582f0-f569-4f72-bd8a-9d0d40d476c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.363328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf582f0-f569-4f72-bd8a-9d0d40d476c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.363618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf582f0-f569-4f72-bd8a-9d0d40d476c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.363897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf582f0-f569-4f72-bd8a-9d0d40d476c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.364662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf582f0-f569-4f72-bd8a-9d0d40d476c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.366257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf582f0-f569-4f72-bd8a-9d0d40d476c5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.370329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 813/1615] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:04 2022] PUT /identity/v3/projects/fc7d718373b94bd3b0c0ff86cd0975bd/users/83bb340d10e44bedad5fc9ddaf332552/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:04.370329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c503658-96d2-48f8-be7e-38bd6ac459b2 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.370329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c503658-96d2-48f8-be7e-38bd6ac459b2 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.370329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c503658-96d2-48f8-be7e-38bd6ac459b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.381515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 803/1616] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:04.381515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf582f0-f569-4f72-bd8a-9d0d40d476c5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.399264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-1c503658-96d2-48f8-be7e-38bd6ac459b2 None None] MFA Rules not processed for user `3552d4b841ad47a2a89d3defa67e521b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:04.400507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1c503658-96d2-48f8-be7e-38bd6ac459b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:04.432982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf582f0-f569-4f72-bd8a-9d0d40d476c5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.440452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:04.449844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:04.464870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 814/1617] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:04.465748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c503658-96d2-48f8-be7e-38bd6ac459b2 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.484179 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40483d80-4d05-4096-8ce7-09a651757761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.484962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483d80-4d05-4096-8ce7-09a651757761 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.485181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483d80-4d05-4096-8ce7-09a651757761 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.485740 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483d80-4d05-4096-8ce7-09a651757761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.532514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c503658-96d2-48f8-be7e-38bd6ac459b2 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.538206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b2ef3-3928-4519-a7aa-1dfe42983108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.539468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b2ef3-3928-4519-a7aa-1dfe42983108 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.539612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b2ef3-3928-4519-a7aa-1dfe42983108 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.540207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b2ef3-3928-4519-a7aa-1dfe42983108 admin admin] PATH_INFO: `/v3/projects/92134d2a18ef41018b6703a7a549fd5d/users/bc435d08003d4328bef2044893980d9d/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.551146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40483d80-4d05-4096-8ce7-09a651757761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.553136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40483d80-4d05-4096-8ce7-09a651757761 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.555284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b2ef3-3928-4519-a7aa-1dfe42983108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92134d2a18ef41018b6703a7a549fd5d, user_id=bc435d08003d4328bef2044893980d9d, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.557967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b2ef3-3928-4519-a7aa-1dfe42983108 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.562729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 804/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:04.564120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40483d80-4d05-4096-8ce7-09a651757761 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.572169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40483d80-4d05-4096-8ce7-09a651757761 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.575976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a21d0a-2031-4053-8e6a-cd37cb4bc06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.576745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a21d0a-2031-4053-8e6a-cd37cb4bc06d service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.577113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a21d0a-2031-4053-8e6a-cd37cb4bc06d service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.577465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a21d0a-2031-4053-8e6a-cd37cb4bc06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.587851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a21d0a-2031-4053-8e6a-cd37cb4bc06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.589298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a21d0a-2031-4053-8e6a-cd37cb4bc06d service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.598480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 805/1619] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:04.599705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3cb84-9789-428d-b8ac-d877ac853f2f None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.599999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3cb84-9789-428d-b8ac-d877ac853f2f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.600273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3cb84-9789-428d-b8ac-d877ac853f2f None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.601565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 806/1620] 10.222.0.23 () {54 vars in 961 bytes} [Mon May 23 15:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:56:04.602269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3cb84-9789-428d-b8ac-d877ac853f2f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.615012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3cb84-9789-428d-b8ac-d877ac853f2f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.619717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd29b1d-3d7b-4eac-b7b4-29bf2b224776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.620399 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd29b1d-3d7b-4eac-b7b4-29bf2b224776 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.620690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd29b1d-3d7b-4eac-b7b4-29bf2b224776 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.620970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd29b1d-3d7b-4eac-b7b4-29bf2b224776 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.621696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd29b1d-3d7b-4eac-b7b4-29bf2b224776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.623301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd29b1d-3d7b-4eac-b7b4-29bf2b224776 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.633991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 807/1621] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:04.635121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 815/1622] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:04 2022] PUT /identity/v3/projects/92134d2a18ef41018b6703a7a549fd5d/users/bc435d08003d4328bef2044893980d9d/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:04.635383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc5402-0fce-4e11-958e-35fb8729bc6b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.640518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc5402-0fce-4e11-958e-35fb8729bc6b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.640518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc5402-0fce-4e11-958e-35fb8729bc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.640848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b2ef3-3928-4519-a7aa-1dfe42983108 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.659127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b7dc5402-0fce-4e11-958e-35fb8729bc6b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:04.660435 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b7dc5402-0fce-4e11-958e-35fb8729bc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:04.706121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b2ef3-3928-4519-a7aa-1dfe42983108 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.711503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033054d-0e83-4444-8c1e-9b0dfc967fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.712428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033054d-0e83-4444-8c1e-9b0dfc967fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.712718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033054d-0e83-4444-8c1e-9b0dfc967fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.713002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033054d-0e83-4444-8c1e-9b0dfc967fb1 admin admin] PATH_INFO: `/v3/projects/523b990b083e468f82994360658a6c3b/users/ab93bf2cebb04d21b49e3bdce650513d/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.715106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033054d-0e83-4444-8c1e-9b0dfc967fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523b990b083e468f82994360658a6c3b, user_id=ab93bf2cebb04d21b49e3bdce650513d, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.718019 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033054d-0e83-4444-8c1e-9b0dfc967fb1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.721960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 808/1623] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:04.723134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995068-8e4a-4865-8754-62b160288a39 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.723352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995068-8e4a-4865-8754-62b160288a39 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.723630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995068-8e4a-4865-8754-62b160288a39 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.724948 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 809/1624] 10.222.0.23 () {54 vars in 961 bytes} [Mon May 23 15:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:56:04.726185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1567e-35b3-4ff8-b623-cf1dfc24b0b5 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.726473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1567e-35b3-4ff8-b623-cf1dfc24b0b5 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.726754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1567e-35b3-4ff8-b623-cf1dfc24b0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.738300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 816/1625] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:04 2022] PUT /identity/v3/projects/523b990b083e468f82994360658a6c3b/users/ab93bf2cebb04d21b49e3bdce650513d/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:04.739268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033054d-0e83-4444-8c1e-9b0dfc967fb1 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.745748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ecd1567e-35b3-4ff8-b623-cf1dfc24b0b5 None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:04.747158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ecd1567e-35b3-4ff8-b623-cf1dfc24b0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:04.807412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:04.811857 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033054d-0e83-4444-8c1e-9b0dfc967fb1 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.817542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3838153-c5e2-432c-b27c-267f44c78942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.818453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3838153-c5e2-432c-b27c-267f44c78942 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.818738 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3838153-c5e2-432c-b27c-267f44c78942 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.819016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3838153-c5e2-432c-b27c-267f44c78942 admin admin] PATH_INFO: `/v3/projects/8a024d1458954153b26bf70f811fa1b2/users/e9210de124314c3ba8dab95c6d3c95a3/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.821090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3838153-c5e2-432c-b27c-267f44c78942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a024d1458954153b26bf70f811fa1b2, user_id=e9210de124314c3ba8dab95c6d3c95a3, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.823746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3838153-c5e2-432c-b27c-267f44c78942 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:04.826108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 810/1626] 10.222.0.23 () {58 vars in 1058 bytes} [Mon May 23 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:04.828028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8430d-e28d-4a20-9adb-2c3837866643 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.828540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8430d-e28d-4a20-9adb-2c3837866643 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.828956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8430d-e28d-4a20-9adb-2c3837866643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.836472 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 817/1627] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:04 2022] PUT /identity/v3/projects/8a024d1458954153b26bf70f811fa1b2/users/e9210de124314c3ba8dab95c6d3c95a3/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:04.837708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3838153-c5e2-432c-b27c-267f44c78942 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:04.840784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b3838153-c5e2-432c-b27c-267f44c78942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:04.873977 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6da8430d-e28d-4a20-9adb-2c3837866643 None None] MFA Rules not processed for user `bc435d08003d4328bef2044893980d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:04.875216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6da8430d-e28d-4a20-9adb-2c3837866643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:04.887449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:04.911549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3838153-c5e2-432c-b27c-267f44c78942 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:04.916644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3997a0-55ba-4db2-924b-75a8ac7cbcb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:04.917535 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3997a0-55ba-4db2-924b-75a8ac7cbcb3 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.917997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3997a0-55ba-4db2-924b-75a8ac7cbcb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.918295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3997a0-55ba-4db2-924b-75a8ac7cbcb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.948562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 811/1628] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:04.950672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564de2f-fc3b-48b4-9298-de7b7c9ccc10 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:04.950672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564de2f-fc3b-48b4-9298-de7b7c9ccc10 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:04.950672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564de2f-fc3b-48b4-9298-de7b7c9ccc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:04.970770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a564de2f-fc3b-48b4-9298-de7b7c9ccc10 None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:04.972154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a564de2f-fc3b-48b4-9298-de7b7c9ccc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:04.998122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3997a0-55ba-4db2-924b-75a8ac7cbcb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:04.999191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3997a0-55ba-4db2-924b-75a8ac7cbcb3 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:05.010203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 818/1629] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:05.019024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0b7ef-ab7c-447b-b129-7dcc1e467149 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.019570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0b7ef-ab7c-447b-b129-7dcc1e467149 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.022042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0b7ef-ab7c-447b-b129-7dcc1e467149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.058855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-9ed0b7ef-ab7c-447b-b129-7dcc1e467149 None None] MFA Rules not processed for user `ab93bf2cebb04d21b49e3bdce650513d`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:05.060055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-9ed0b7ef-ab7c-447b-b129-7dcc1e467149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:05.072286 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 812/1630] 10.222.0.23 () {58 vars in 1058 bytes} [Mon May 23 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:05.072981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564de2f-fc3b-48b4-9298-de7b7c9ccc10 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:05.083884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564de2f-fc3b-48b4-9298-de7b7c9ccc10 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:05.089705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db0e11-c915-47cd-b79c-1b96e86cfca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:05.090531 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db0e11-c915-47cd-b79c-1b96e86cfca0 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.090668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db0e11-c915-47cd-b79c-1b96e86cfca0 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.090925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db0e11-c915-47cd-b79c-1b96e86cfca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.137424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 819/1631] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:05.138882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c0aae-fc2f-442a-bace-90d9de5bb30b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.139239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c0aae-fc2f-442a-bace-90d9de5bb30b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.139591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c0aae-fc2f-442a-bace-90d9de5bb30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.179934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e99c0aae-fc2f-442a-bace-90d9de5bb30b None None] MFA Rules not processed for user `e9210de124314c3ba8dab95c6d3c95a3`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:05.181409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e99c0aae-fc2f-442a-bace-90d9de5bb30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:05.184120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db0e11-c915-47cd-b79c-1b96e86cfca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:05.186390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db0e11-c915-47cd-b79c-1b96e86cfca0 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:05.191826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:05.201625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 813/1632] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:05.202755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db0e11-c915-47cd-b79c-1b96e86cfca0 service nova] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:05.214423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:05.217102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db0e11-c915-47cd-b79c-1b96e86cfca0 service nova] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:05.227630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11cb1c-2bee-4605-abde-7f83f3b48841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:05.228330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11cb1c-2bee-4605-abde-7f83f3b48841 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.228619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11cb1c-2bee-4605-abde-7f83f3b48841 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.228900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11cb1c-2bee-4605-abde-7f83f3b48841 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.229593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11cb1c-2bee-4605-abde-7f83f3b48841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:05.231499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11cb1c-2bee-4605-abde-7f83f3b48841 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:05.268851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 820/1633] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:05.275341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c0aae-fc2f-442a-bace-90d9de5bb30b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:05.290898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c0aae-fc2f-442a-bace-90d9de5bb30b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:05.296592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb004d00-bece-4646-b714-9687c5e273da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:05.310613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb004d00-bece-4646-b714-9687c5e273da service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.310613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb004d00-bece-4646-b714-9687c5e273da service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.310613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb004d00-bece-4646-b714-9687c5e273da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.347819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 814/1634] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:05 2022] POST /identity/v3/projects => generated 403 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:05.360854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11cb1c-2bee-4605-abde-7f83f3b48841 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:05.424344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb004d00-bece-4646-b714-9687c5e273da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:05.425990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb004d00-bece-4646-b714-9687c5e273da service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:05.445941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 821/1635] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:05.446884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb004d00-bece-4646-b714-9687c5e273da service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:05.495991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11cb1c-2bee-4605-abde-7f83f3b48841 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:05.499960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a990e-6107-4dd0-b9b4-d84f368602bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:05.500664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a990e-6107-4dd0-b9b4-d84f368602bc service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.500967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a990e-6107-4dd0-b9b4-d84f368602bc service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.501241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a990e-6107-4dd0-b9b4-d84f368602bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.535768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb004d00-bece-4646-b714-9687c5e273da service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:05.543005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e7eae-f3f8-40e3-8ddb-897de398c8d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:05.543005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e7eae-f3f8-40e3-8ddb-897de398c8d8 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.543005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e7eae-f3f8-40e3-8ddb-897de398c8d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.543005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e7eae-f3f8-40e3-8ddb-897de398c8d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.618360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a990e-6107-4dd0-b9b4-d84f368602bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:05.619834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e7eae-f3f8-40e3-8ddb-897de398c8d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:05.620866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a990e-6107-4dd0-b9b4-d84f368602bc service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:05.621302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e7eae-f3f8-40e3-8ddb-897de398c8d8 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:05.631613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 822/1636] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:05.632211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e7eae-f3f8-40e3-8ddb-897de398c8d8 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:05.636872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 815/1637] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:05.655646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde225d-72c9-4fc7-9df0-2161e28eb981 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.658673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:05.659466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde225d-72c9-4fc7-9df0-2161e28eb981 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.659882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde225d-72c9-4fc7-9df0-2161e28eb981 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.663738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 816/1638] 10.222.0.23 () {54 vars in 912 bytes} [Mon May 23 15:56:05 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:56:05.671365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9282-1118-4cc9-a680-4e6f6cf8ffed None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.671808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9282-1118-4cc9-a680-4e6f6cf8ffed None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.672218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9282-1118-4cc9-a680-4e6f6cf8ffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.710587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e7eae-f3f8-40e3-8ddb-897de398c8d8 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:05.725174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770c62a-8cc1-475e-bb8b-11e4cb2d282f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:05.725874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770c62a-8cc1-475e-bb8b-11e4cb2d282f admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.726806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770c62a-8cc1-475e-bb8b-11e4cb2d282f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.726806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770c62a-8cc1-475e-bb8b-11e4cb2d282f admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.727723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770c62a-8cc1-475e-bb8b-11e4cb2d282f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:05.730772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4770c62a-8cc1-475e-bb8b-11e4cb2d282f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:05.769365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1b7f9282-1118-4cc9-a680-4e6f6cf8ffed None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:05.771215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1b7f9282-1118-4cc9-a680-4e6f6cf8ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:05.875303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 823/1639] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:05 2022] POST /identity/v3/users => generated 308 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:05.875303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770c62a-8cc1-475e-bb8b-11e4cb2d282f admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:05.910061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770c62a-8cc1-475e-bb8b-11e4cb2d282f admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:05.926804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:05.928970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:05.929274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:05.929855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] PATH_INFO: `/v3/users/0483214f1ab6452e92f627dffbbd5d44` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:05.950424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0483214f1ab6452e92f627dffbbd5d44)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:05.951632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: WARNING py.warnings [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:56:05.951632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: warnings.warn(msg) May 23 15:56:05.951632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: May 23 15:56:05.958814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:06.024905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 817/1640] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 15:56:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:06.026741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8917652-51c3-456a-a512-0caef9b431d7 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.027094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8917652-51c3-456a-a512-0caef9b431d7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.027436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8917652-51c3-456a-a512-0caef9b431d7 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.035396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 818/1641] 10.222.0.23 () {54 vars in 912 bytes} [Mon May 23 15:56:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:56:06.036440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8917652-51c3-456a-a512-0caef9b431d7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:06.052863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8917652-51c3-456a-a512-0caef9b431d7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:06.057708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c2edf-e7d6-4f7b-9a54-633f6a0e7a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:06.058492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c2edf-e7d6-4f7b-9a54-633f6a0e7a7a admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.058877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c2edf-e7d6-4f7b-9a54-633f6a0e7a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.059215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c2edf-e7d6-4f7b-9a54-633f6a0e7a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.059941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c2edf-e7d6-4f7b-9a54-633f6a0e7a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:06.061551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c2edf-e7d6-4f7b-9a54-633f6a0e7a7a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:06.073297 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 819/1642] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:06.077202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c2edf-e7d6-4f7b-9a54-633f6a0e7a7a admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:06.087644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c2edf-e7d6-4f7b-9a54-633f6a0e7a7a admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:06.091347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3751148-94a6-4455-99c1-5c96402df3aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:06.092081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3751148-94a6-4455-99c1-5c96402df3aa service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.092626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3751148-94a6-4455-99c1-5c96402df3aa service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.092626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3751148-94a6-4455-99c1-5c96402df3aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.101563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:06.105859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3751148-94a6-4455-99c1-5c96402df3aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:06.107332 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3751148-94a6-4455-99c1-5c96402df3aa service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:06.119531 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 820/1643] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:06.120751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7b9bb-461b-4b24-b9f5-99e8f5176b92 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.121522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7b9bb-461b-4b24-b9f5-99e8f5176b92 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.121522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7b9bb-461b-4b24-b9f5-99e8f5176b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.146566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a9a7b9bb-461b-4b24-b9f5-99e8f5176b92 None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:06.146566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a9a7b9bb-461b-4b24-b9f5-99e8f5176b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:06.156943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 821/1644] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 15:56:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:06.157814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7b9bb-461b-4b24-b9f5-99e8f5176b92 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:06.170316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7b9bb-461b-4b24-b9f5-99e8f5176b92 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:06.176221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae96fe7-6617-405e-ad32-2bf1a1853935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:06.177173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae96fe7-6617-405e-ad32-2bf1a1853935 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.177564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae96fe7-6617-405e-ad32-2bf1a1853935 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.177756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae96fe7-6617-405e-ad32-2bf1a1853935 admin admin] PATH_INFO: `/v3/projects/f2bad68b6b024474b340ee4ce95b7156/users/89312bcc7397444994e5b5b4d53c5208/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.205654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae96fe7-6617-405e-ad32-2bf1a1853935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2bad68b6b024474b340ee4ce95b7156, user_id=89312bcc7397444994e5b5b4d53c5208, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:06.213058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae96fe7-6617-405e-ad32-2bf1a1853935 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:06.255566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0483214f1ab6452e92f627dffbbd5d44'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:06.256946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0483214f1ab6452e92f627dffbbd5d44'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:06.257768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0483214f1ab6452e92f627dffbbd5d44'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:06.270531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0483214f1ab6452e92f627dffbbd5d44'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:06.292127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0483214f1ab6452e92f627dffbbd5d44'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:06.327770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 822/1645] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:06 2022] PUT /identity/v3/projects/f2bad68b6b024474b340ee4ce95b7156/users/89312bcc7397444994e5b5b4d53c5208/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:06.327770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae96fe7-6617-405e-ad32-2bf1a1853935 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:06.329168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7ae96fe7-6617-405e-ad32-2bf1a1853935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:06.403812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae96fe7-6617-405e-ad32-2bf1a1853935 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:06.408499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c4dea-9a02-4de6-ac12-5fc2ebdbe07d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:06.417176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c4dea-9a02-4de6-ac12-5fc2ebdbe07d service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.417176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c4dea-9a02-4de6-ac12-5fc2ebdbe07d service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.417176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c4dea-9a02-4de6-ac12-5fc2ebdbe07d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.417176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-fe5c4dea-9a02-4de6-ac12-5fc2ebdbe07d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:06.417533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 824/1646] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:05 2022] DELETE /identity/v3/users/0483214f1ab6452e92f627dffbbd5d44 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:06.417533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:06.418036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:06.534817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67310f-749a-4ea6-a955-27433fca022a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:06.536873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c4dea-9a02-4de6-ac12-5fc2ebdbe07d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:06.537945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c4dea-9a02-4de6-ac12-5fc2ebdbe07d service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:06.542016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc3edd-1642-4c6a-adac-9e32bf8f481a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:06.542827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc3edd-1642-4c6a-adac-9e32bf8f481a admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.543215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc3edd-1642-4c6a-adac-9e32bf8f481a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.543594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc3edd-1642-4c6a-adac-9e32bf8f481a admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.545038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc3edd-1642-4c6a-adac-9e32bf8f481a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:06.547479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc3edd-1642-4c6a-adac-9e32bf8f481a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:06.551829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:06.555999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 823/1647] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:06.557514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e5491-89fb-4a43-90f4-2f6cedd67e69 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.558539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e5491-89fb-4a43-90f4-2f6cedd67e69 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.558539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e5491-89fb-4a43-90f4-2f6cedd67e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.564404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 825/1648] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:06.567562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc3edd-1642-4c6a-adac-9e32bf8f481a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:06.583798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc3edd-1642-4c6a-adac-9e32bf8f481a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:06.601203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7ce1c-9274-48d6-88c3-9519ab5b7a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:06.602294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ce1c-9274-48d6-88c3-9519ab5b7a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.602722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ce1c-9274-48d6-88c3-9519ab5b7a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.603104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ce1c-9274-48d6-88c3-9519ab5b7a2a admin admin] PATH_INFO: `/v3/projects/f2bad68b6b024474b340ee4ce95b7156/users/89312bcc7397444994e5b5b4d53c5208/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.605186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-777e5491-89fb-4a43-90f4-2f6cedd67e69 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:06.611272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-777e5491-89fb-4a43-90f4-2f6cedd67e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:06.611380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7ce1c-9274-48d6-88c3-9519ab5b7a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2bad68b6b024474b340ee4ce95b7156, user_id=89312bcc7397444994e5b5b4d53c5208, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:06.614503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7ce1c-9274-48d6-88c3-9519ab5b7a2a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:06.633559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 824/1649] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:06.633559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e91eb1-cbb1-40fc-9cb1-3d3dfd951ce6 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.633559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e91eb1-cbb1-40fc-9cb1-3d3dfd951ce6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.633559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e91eb1-cbb1-40fc-9cb1-3d3dfd951ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.636739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 826/1650] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:06 2022] PUT /identity/v3/projects/f2bad68b6b024474b340ee4ce95b7156/users/89312bcc7397444994e5b5b4d53c5208/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:06.647164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e813cfa-b3f2-4a37-be4f-ecce039f060f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.647460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e813cfa-b3f2-4a37-be4f-ecce039f060f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.647737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e813cfa-b3f2-4a37-be4f-ecce039f060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.705521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-03e91eb1-cbb1-40fc-9cb1-3d3dfd951ce6 None None] MFA Rules not processed for user `c2075a6ceede424087448411badff88b`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:06.706786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-03e91eb1-cbb1-40fc-9cb1-3d3dfd951ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:06.723267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-3e813cfa-b3f2-4a37-be4f-ecce039f060f None None] MFA Rules not processed for user `89312bcc7397444994e5b5b4d53c5208`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:06.726376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3e813cfa-b3f2-4a37-be4f-ecce039f060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:06.800531 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:06.834089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 825/1651] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:06.835660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e91eb1-cbb1-40fc-9cb1-3d3dfd951ce6 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:06.847689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-03e91eb1-cbb1-40fc-9cb1-3d3dfd951ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:06.893800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 827/1652] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:06 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:06.895342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ce914-c17b-4b68-823e-0e69bfdb8536 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.897133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ce914-c17b-4b68-823e-0e69bfdb8536 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.897133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ce914-c17b-4b68-823e-0e69bfdb8536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:06.935508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-8e6ce914-c17b-4b68-823e-0e69bfdb8536 None None] MFA Rules not processed for user `c2075a6ceede424087448411badff88b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:06.942698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8e6ce914-c17b-4b68-823e-0e69bfdb8536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:06.962024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 828/1653] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:06.973075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e91eb1-cbb1-40fc-9cb1-3d3dfd951ce6 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:06.979049 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1f803-6d65-4f2c-b80e-65ea08b0f4ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:06.979944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1f803-6d65-4f2c-b80e-65ea08b0f4ce service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:06.980452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1f803-6d65-4f2c-b80e-65ea08b0f4ce service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:06.980884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1f803-6d65-4f2c-b80e-65ea08b0f4ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:07.051532 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1f803-6d65-4f2c-b80e-65ea08b0f4ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:07.054839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1f803-6d65-4f2c-b80e-65ea08b0f4ce service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:07.065147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 826/1654] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:07.543807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ce914-c17b-4b68-823e-0e69bfdb8536 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:07.545525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8e6ce914-c17b-4b68-823e-0e69bfdb8536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:07.613199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ce914-c17b-4b68-823e-0e69bfdb8536 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:07.616771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2e932-db0d-4776-9b66-3ab38dad1629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:07.617431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2e932-db0d-4776-9b66-3ab38dad1629 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:07.617717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2e932-db0d-4776-9b66-3ab38dad1629 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:07.617995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2e932-db0d-4776-9b66-3ab38dad1629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:07.644925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2e932-db0d-4776-9b66-3ab38dad1629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:07.646922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2e932-db0d-4776-9b66-3ab38dad1629 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:07.659560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 829/1655] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:07 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:07.876974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1f803-6d65-4f2c-b80e-65ea08b0f4ce service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:07.876974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9ad1f803-6d65-4f2c-b80e-65ea08b0f4ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:08.019826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1f803-6d65-4f2c-b80e-65ea08b0f4ce service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:08.037006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4268653-5476-49be-8a37-3bf201537145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:08.038438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4268653-5476-49be-8a37-3bf201537145 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:08.038729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4268653-5476-49be-8a37-3bf201537145 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:08.039002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4268653-5476-49be-8a37-3bf201537145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:08.087802 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4268653-5476-49be-8a37-3bf201537145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:08.089966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4268653-5476-49be-8a37-3bf201537145 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:08.115704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 827/1656] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:08.179694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2e932-db0d-4776-9b66-3ab38dad1629 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:08.195166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2e932-db0d-4776-9b66-3ab38dad1629 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:08.206533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:08.207610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:08.209264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:08.213775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] PATH_INFO: `/v3/users/bc435d08003d4328bef2044893980d9d` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:08.214517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc435d08003d4328bef2044893980d9d)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:08.218640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:08.251294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4268653-5476-49be-8a37-3bf201537145 service nova] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:08.255438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d4268653-5476-49be-8a37-3bf201537145 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:08.296780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4268653-5476-49be-8a37-3bf201537145 service nova] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:08.315701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:08.316829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:08.317112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:08.317390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] PATH_INFO: `/v3/projects/b916e4deb43a43e7a8379aa285d3bdd6` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:08.318747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b916e4deb43a43e7a8379aa285d3bdd6)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:08.319947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: WARNING py.warnings [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 15:56:08.319947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: warnings.warn(msg) May 23 15:56:08.319947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: May 23 15:56:08.321196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:08.349201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:08.534953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc435d08003d4328bef2044893980d9d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:08.536132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc435d08003d4328bef2044893980d9d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:08.547850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc435d08003d4328bef2044893980d9d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:08.550114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b916e4deb43a43e7a8379aa285d3bdd6', 'request_id': 'req-6cf5ca34-303e-40e7-9053-ec6144d3f779'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:08.552845 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc435d08003d4328bef2044893980d9d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:08.560323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:08.575281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-006c8f90-f4de-481a-9016-ed8b001d7dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc435d08003d4328bef2044893980d9d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:08.585294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b916e4deb43a43e7a8379aa285d3bdd6', 'request_id': 'req-6cf5ca34-303e-40e7-9053-ec6144d3f779'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:08.600539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b916e4deb43a43e7a8379aa285d3bdd6', 'request_id': 'req-6cf5ca34-303e-40e7-9053-ec6144d3f779'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:08.678370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 830/1657] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:08 2022] DELETE /identity/v3/users/bc435d08003d4328bef2044893980d9d => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:08.693064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fb19f-4c9f-4132-9b93-5ec0b53efcc9 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:08.693408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fb19f-4c9f-4132-9b93-5ec0b53efcc9 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:08.693701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fb19f-4c9f-4132-9b93-5ec0b53efcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:08.726662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2d6fb19f-4c9f-4132-9b93-5ec0b53efcc9 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:08.727841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2d6fb19f-4c9f-4132-9b93-5ec0b53efcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:08.732918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] The token cache is being invalidate because project b916e4deb43a43e7a8379aa285d3bdd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:08.733216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:08.733892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:08.743309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 828/1658] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:08 2022] DELETE /identity/v3/projects/b916e4deb43a43e7a8379aa285d3bdd6 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:08.746373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:08.747287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:08.837650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 831/1659] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:08.839222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5817f1f-04ec-4b4e-b005-74190b3fcb14 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:08.839521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5817f1f-04ec-4b4e-b005-74190b3fcb14 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:08.839806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5817f1f-04ec-4b4e-b005-74190b3fcb14 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:08.846300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 832/1660] 10.222.0.23 () {54 vars in 912 bytes} [Mon May 23 15:56:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:56:08.847892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5817f1f-04ec-4b4e-b005-74190b3fcb14 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:08.849956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e5817f1f-04ec-4b4e-b005-74190b3fcb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:08.857549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5ca34-303e-40e7-9053-ec6144d3f779 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:08.867581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:08.868695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:08.869081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:08.869454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] PATH_INFO: `/v3/users/f32d1bd8cad44434a6bd04bf04170d28` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:08.871093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32d1bd8cad44434a6bd04bf04170d28)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:08.871437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5817f1f-04ec-4b4e-b005-74190b3fcb14 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:08.873598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:08.877761 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:08.878863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:08.879160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:08.879579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] PATH_INFO: `/v3/users/ab93bf2cebb04d21b49e3bdce650513d` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:08.886927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab93bf2cebb04d21b49e3bdce650513d)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:08.889766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:09.167556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32d1bd8cad44434a6bd04bf04170d28'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:09.168261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32d1bd8cad44434a6bd04bf04170d28'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:09.182514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab93bf2cebb04d21b49e3bdce650513d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:09.187570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab93bf2cebb04d21b49e3bdce650513d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:09.187570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab93bf2cebb04d21b49e3bdce650513d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:09.187854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32d1bd8cad44434a6bd04bf04170d28'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:09.187854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32d1bd8cad44434a6bd04bf04170d28'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:09.196597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab93bf2cebb04d21b49e3bdce650513d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:09.200124 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-557df96f-dd2c-4cef-adb4-fffecb99315b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32d1bd8cad44434a6bd04bf04170d28'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:09.216764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab93bf2cebb04d21b49e3bdce650513d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:09.316918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 829/1661] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:08 2022] DELETE /identity/v3/users/f32d1bd8cad44434a6bd04bf04170d28 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:09.318209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06be722-ceb8-435e-81ca-f8be488887b8 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:09.318523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06be722-ceb8-435e-81ca-f8be488887b8 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:09.318780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06be722-ceb8-435e-81ca-f8be488887b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:09.372858 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a06be722-ceb8-435e-81ca-f8be488887b8 None None] MFA Rules not processed for user `a39f5189d63d467dbd914f553c261c01`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:09.374321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a06be722-ceb8-435e-81ca-f8be488887b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:09.381795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:09.400315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 833/1662] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:08 2022] DELETE /identity/v3/users/ab93bf2cebb04d21b49e3bdce650513d => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:09.401441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:09.415789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:09.458591 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 830/1663] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 15:56:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:09.461166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04d4fb-1f8c-4658-922c-c4570ef9d60a None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:09.462559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04d4fb-1f8c-4658-922c-c4570ef9d60a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:09.462956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04d4fb-1f8c-4658-922c-c4570ef9d60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:09.502194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8b04d4fb-1f8c-4658-922c-c4570ef9d60a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:09.505362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8b04d4fb-1f8c-4658-922c-c4570ef9d60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:09.631063 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fb3d1-a645-485c-8329-6782aa2ba2e2 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:09.633263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 831/1664] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:09.633974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04d4fb-1f8c-4658-922c-c4570ef9d60a None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:09.660818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb02c1f-2719-4f26-8dc4-ec16b6a91483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:09.663296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02c1f-2719-4f26-8dc4-ec16b6a91483 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:09.664723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04d4fb-1f8c-4658-922c-c4570ef9d60a None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:09.665783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02c1f-2719-4f26-8dc4-ec16b6a91483 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:09.668902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02c1f-2719-4f26-8dc4-ec16b6a91483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:09.677491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7eb02c1f-2719-4f26-8dc4-ec16b6a91483 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:09.690791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6fffe-ddb9-46ad-926c-147c6d2e4d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:09.692755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6fffe-ddb9-46ad-926c-147c6d2e4d9d service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:09.693047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6fffe-ddb9-46ad-926c-147c6d2e4d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:09.693326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6fffe-ddb9-46ad-926c-147c6d2e4d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:09.714746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6fffe-ddb9-46ad-926c-147c6d2e4d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:09.716420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6fffe-ddb9-46ad-926c-147c6d2e4d9d service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:09.723987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb02c1f-2719-4f26-8dc4-ec16b6a91483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:09.727541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb02c1f-2719-4f26-8dc4-ec16b6a91483 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:09.733857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 832/1665] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:09.734748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6fffe-ddb9-46ad-926c-147c6d2e4d9d service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:09.738532 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-53f6fffe-ddb9-46ad-926c-147c6d2e4d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:09.763439 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6fffe-ddb9-46ad-926c-147c6d2e4d9d service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:09.774837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bfa13-bfa5-402a-b450-71ffa634b00b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:09.775525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bfa13-bfa5-402a-b450-71ffa634b00b service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:09.775813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bfa13-bfa5-402a-b450-71ffa634b00b service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:09.776095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bfa13-bfa5-402a-b450-71ffa634b00b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:09.798500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 834/1666] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:09.807721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bfa13-bfa5-402a-b450-71ffa634b00b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:09.809238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bfa13-bfa5-402a-b450-71ffa634b00b service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:09.829878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 833/1667] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:09.999708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:10.452126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb02c1f-2719-4f26-8dc4-ec16b6a91483 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:10.452582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7eb02c1f-2719-4f26-8dc4-ec16b6a91483 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:10.498130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb02c1f-2719-4f26-8dc4-ec16b6a91483 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:10.528101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:10.529275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:10.531971 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:10.533003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] PATH_INFO: `/v3/users/b3f3a9a74bb543d494845c394fe0c55f` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:10.540950 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f3a9a74bb543d494845c394fe0c55f)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:10.543120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:10.633983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866bd4-e784-46da-a69a-da7d48e355e3 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:10.635123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866bd4-e784-46da-a69a-da7d48e355e3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:10.635588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b866bd4-e784-46da-a69a-da7d48e355e3 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:10.637888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 834/1668] 10.222.0.23 () {54 vars in 960 bytes} [Mon May 23 15:56:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:56:10.645097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942044fd-3fa0-48bc-9527-bb557fb662e0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:10.645625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942044fd-3fa0-48bc-9527-bb557fb662e0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:10.646322 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942044fd-3fa0-48bc-9527-bb557fb662e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:10.674474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-942044fd-3fa0-48bc-9527-bb557fb662e0 None None] MFA Rules not processed for user `312ed2b555424c6a96b3d972ffea5ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:10.675952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-942044fd-3fa0-48bc-9527-bb557fb662e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:10.778349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 835/1669] 10.222.0.23 () {58 vars in 1057 bytes} [Mon May 23 15:56:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:10.787720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942044fd-3fa0-48bc-9527-bb557fb662e0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:10.806768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942044fd-3fa0-48bc-9527-bb557fb662e0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:10.814794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512eaa5-f954-4186-8ded-4cd1dd7a72ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:10.815862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512eaa5-f954-4186-8ded-4cd1dd7a72ef service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:10.816517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512eaa5-f954-4186-8ded-4cd1dd7a72ef service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:10.816988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512eaa5-f954-4186-8ded-4cd1dd7a72ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:10.820681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5512eaa5-f954-4186-8ded-4cd1dd7a72ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:10.824075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f3a9a74bb543d494845c394fe0c55f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:10.824918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f3a9a74bb543d494845c394fe0c55f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:10.826501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f3a9a74bb543d494845c394fe0c55f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:10.841814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512eaa5-f954-4186-8ded-4cd1dd7a72ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:10.843947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512eaa5-f954-4186-8ded-4cd1dd7a72ef service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:10.845958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f3a9a74bb543d494845c394fe0c55f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:10.875565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 836/1670] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:56:10.876154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7695b46f-a905-4bbb-a1ca-3b2ea695df18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f3a9a74bb543d494845c394fe0c55f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:10.891650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512eaa5-f954-4186-8ded-4cd1dd7a72ef service placement] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:10.894962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5512eaa5-f954-4186-8ded-4cd1dd7a72ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:10.921285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512eaa5-f954-4186-8ded-4cd1dd7a72ef service placement] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:10.935446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:10.936344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:10.936737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:10.937007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] PATH_INFO: `/v3/projects/92134d2a18ef41018b6703a7a549fd5d` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:10.939882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92134d2a18ef41018b6703a7a549fd5d)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:10.941894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:10.975950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:11.027044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 835/1671] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:10 2022] DELETE /identity/v3/users/b3f3a9a74bb543d494845c394fe0c55f => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:11.031646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5db92-737a-4bb9-95ac-ed96a0439c9a None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:11.031939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5db92-737a-4bb9-95ac-ed96a0439c9a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:11.032228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5db92-737a-4bb9-95ac-ed96a0439c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:11.072293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e2d5db92-737a-4bb9-95ac-ed96a0439c9a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:11.076769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e2d5db92-737a-4bb9-95ac-ed96a0439c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:11.148204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92134d2a18ef41018b6703a7a549fd5d', 'request_id': 'req-6e6f678d-54ab-4034-824a-398c286c8f31'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.169454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92134d2a18ef41018b6703a7a549fd5d', 'request_id': 'req-6e6f678d-54ab-4034-824a-398c286c8f31'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.186117 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92134d2a18ef41018b6703a7a549fd5d', 'request_id': 'req-6e6f678d-54ab-4034-824a-398c286c8f31'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.244911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 836/1672] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:11.246817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5db92-737a-4bb9-95ac-ed96a0439c9a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:11.248548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e2d5db92-737a-4bb9-95ac-ed96a0439c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:11.274281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5db92-737a-4bb9-95ac-ed96a0439c9a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:11.309778 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] The token cache is being invalidate because project 92134d2a18ef41018b6703a7a549fd5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:11.310072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.311724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.317747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 837/1673] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:10 2022] DELETE /identity/v3/projects/92134d2a18ef41018b6703a7a549fd5d => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:11.318471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:11.325843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:11.509672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f678d-54ab-4034-824a-398c286c8f31 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:11.514134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49ff4-c755-4222-90b2-8a85a2bb318a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:11.527006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49ff4-c755-4222-90b2-8a85a2bb318a service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:11.527006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49ff4-c755-4222-90b2-8a85a2bb318a service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:11.527006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49ff4-c755-4222-90b2-8a85a2bb318a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:11.527724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:11.528169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a4a49ff4-c755-4222-90b2-8a85a2bb318a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:11.528322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:11.529473 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:11.529946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] PATH_INFO: `/v3/projects/523b990b083e468f82994360658a6c3b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:11.531406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=523b990b083e468f82994360658a6c3b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:11.533746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a49ff4-c755-4222-90b2-8a85a2bb318a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:11.535259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:11.535372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a49ff4-c755-4222-90b2-8a85a2bb318a service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:11.546529 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 838/1674] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:11.547356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49ff4-c755-4222-90b2-8a85a2bb318a service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:11.548874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a4a49ff4-c755-4222-90b2-8a85a2bb318a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:11.562569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49ff4-c755-4222-90b2-8a85a2bb318a service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:11.569214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:11.570419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:11.571120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:11.571370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] PATH_INFO: `/v3/users/89312bcc7397444994e5b5b4d53c5208` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:11.573656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89312bcc7397444994e5b5b4d53c5208)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:11.576147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:11.665237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523b990b083e468f82994360658a6c3b', 'request_id': 'req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.674093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523b990b083e468f82994360658a6c3b', 'request_id': 'req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.681269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523b990b083e468f82994360658a6c3b', 'request_id': 'req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.845993 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] The token cache is being invalidate because project 523b990b083e468f82994360658a6c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:11.846624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.846963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.853809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 837/1675] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:11 2022] DELETE /identity/v3/projects/523b990b083e468f82994360658a6c3b => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:11.856984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:11.860007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:11.862306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89312bcc7397444994e5b5b4d53c5208'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.866850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89312bcc7397444994e5b5b4d53c5208'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.896830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89312bcc7397444994e5b5b4d53c5208'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.897451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89312bcc7397444994e5b5b4d53c5208'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.913489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10d94114-3429-45f2-8a32-f8ee534943e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89312bcc7397444994e5b5b4d53c5208'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:11.967675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75da13-6ffa-4b55-a4ef-5c8958054e5f admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:11.973001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:11.973957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:11.974310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:11.974669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] PATH_INFO: `/v3/projects/cb224c26e34943d996918e3762c1e965` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:11.976562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb224c26e34943d996918e3762c1e965)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:11.979040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:12.011211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:12.044576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 839/1676] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:11 2022] DELETE /identity/v3/users/89312bcc7397444994e5b5b4d53c5208 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:12.044576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51505633-ecbf-4c82-a314-6c1bbc9bf134 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:12.044576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51505633-ecbf-4c82-a314-6c1bbc9bf134 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:12.044576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51505633-ecbf-4c82-a314-6c1bbc9bf134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:12.081605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-51505633-ecbf-4c82-a314-6c1bbc9bf134 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:12.082919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-51505633-ecbf-4c82-a314-6c1bbc9bf134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:12.092882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:12.140200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb224c26e34943d996918e3762c1e965', 'request_id': 'req-ea1662c5-ee95-4bda-ab47-e4ef42066b81'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.144401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 840/1677] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:12.149373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb224c26e34943d996918e3762c1e965', 'request_id': 'req-ea1662c5-ee95-4bda-ab47-e4ef42066b81'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.151910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234426f4-563a-4bf9-9bba-7cec42d7102e None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:12.162600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234426f4-563a-4bf9-9bba-7cec42d7102e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:12.165913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234426f4-563a-4bf9-9bba-7cec42d7102e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:12.176544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb224c26e34943d996918e3762c1e965', 'request_id': 'req-ea1662c5-ee95-4bda-ab47-e4ef42066b81'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.212324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-234426f4-563a-4bf9-9bba-7cec42d7102e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:12.213899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-234426f4-563a-4bf9-9bba-7cec42d7102e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:12.299335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] The token cache is being invalidate because project cb224c26e34943d996918e3762c1e965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:12.299964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.301074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.307799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 838/1678] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:11 2022] DELETE /identity/v3/projects/cb224c26e34943d996918e3762c1e965 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:12.309458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:12.312303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:12.332653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 841/1679] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:12.337168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234426f4-563a-4bf9-9bba-7cec42d7102e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:12.342729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-234426f4-563a-4bf9-9bba-7cec42d7102e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:12.372561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234426f4-563a-4bf9-9bba-7cec42d7102e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:12.382030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:12.384084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:12.384625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:12.385068 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] PATH_INFO: `/v3/users/e9210de124314c3ba8dab95c6d3c95a3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:12.386536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9210de124314c3ba8dab95c6d3c95a3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:12.388920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:12.457775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1662c5-ee95-4bda-ab47-e4ef42066b81 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:12.468820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:12.470404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:12.470894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:12.471370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] PATH_INFO: `/v3/projects/453b7ba3cabe4e2b805865ea2c8945bc` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:12.473159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=453b7ba3cabe4e2b805865ea2c8945bc)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:12.481106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:12.679686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453b7ba3cabe4e2b805865ea2c8945bc', 'request_id': 'req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.682729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9210de124314c3ba8dab95c6d3c95a3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.684054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9210de124314c3ba8dab95c6d3c95a3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.697013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453b7ba3cabe4e2b805865ea2c8945bc', 'request_id': 'req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.708416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453b7ba3cabe4e2b805865ea2c8945bc', 'request_id': 'req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.712624 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9210de124314c3ba8dab95c6d3c95a3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.713658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9210de124314c3ba8dab95c6d3c95a3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.725264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-92dcfdf2-5c0e-4633-8def-340a3e26d9ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9210de124314c3ba8dab95c6d3c95a3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.791889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] The token cache is being invalidate because project 453b7ba3cabe4e2b805865ea2c8945bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:12.792304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.793553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:12.800005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 839/1680] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:12 2022] DELETE /identity/v3/projects/453b7ba3cabe4e2b805865ea2c8945bc => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:12.801375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:12.803724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:12.879042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 842/1681] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:12 2022] DELETE /identity/v3/users/e9210de124314c3ba8dab95c6d3c95a3 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:12.880501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bc284-cbb1-4a81-95e5-1eda2cc6eee0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:12.880801 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bc284-cbb1-4a81-95e5-1eda2cc6eee0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:12.881128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bc284-cbb1-4a81-95e5-1eda2cc6eee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:12.902727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-991bc284-cbb1-4a81-95e5-1eda2cc6eee0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:12.903973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-991bc284-cbb1-4a81-95e5-1eda2cc6eee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:13.014731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a4ae5-86e8-46d8-ac56-56ab522834b4 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:13.035164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 843/1682] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:13.045769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c5ea2-4931-45ca-ad4d-34a6dd3a9e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:13.046079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dbec6-0f5f-48e9-bec2-fa2a3a5c3fce None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:13.046387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dbec6-0f5f-48e9-bec2-fa2a3a5c3fce None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:13.046669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dbec6-0f5f-48e9-bec2-fa2a3a5c3fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:13.048082 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c5ea2-4931-45ca-ad4d-34a6dd3a9e6a admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:13.048388 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c5ea2-4931-45ca-ad4d-34a6dd3a9e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:13.048748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c5ea2-4931-45ca-ad4d-34a6dd3a9e6a admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:13.052669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c5ea2-4931-45ca-ad4d-34a6dd3a9e6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:13.053933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c5ea2-4931-45ca-ad4d-34a6dd3a9e6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:13.059512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c5ea2-4931-45ca-ad4d-34a6dd3a9e6a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:13.097260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ff6dbec6-0f5f-48e9-bec2-fa2a3a5c3fce None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:13.098871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ff6dbec6-0f5f-48e9-bec2-fa2a3a5c3fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:13.104921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 840/1683] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:13.107858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c5ea2-4931-45ca-ad4d-34a6dd3a9e6a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:13.119864 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 844/1684] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:13.123872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dbec6-0f5f-48e9-bec2-fa2a3a5c3fce None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:13.129695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c5ea2-4931-45ca-ad4d-34a6dd3a9e6a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:13.144428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dbec6-0f5f-48e9-bec2-fa2a3a5c3fce None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:13.145419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1204d4-64b0-4845-8a9b-08a2d4d9826e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:13.146538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1204d4-64b0-4845-8a9b-08a2d4d9826e admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:13.152593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1204d4-64b0-4845-8a9b-08a2d4d9826e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:13.152593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1204d4-64b0-4845-8a9b-08a2d4d9826e admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:13.152593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1204d4-64b0-4845-8a9b-08a2d4d9826e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:13.154155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1204d4-64b0-4845-8a9b-08a2d4d9826e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:13.158750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b9d5c-6fd9-413d-9792-cff7211d41f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:13.159639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9d5c-6fd9-413d-9792-cff7211d41f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:13.160470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9d5c-6fd9-413d-9792-cff7211d41f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:13.160832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9d5c-6fd9-413d-9792-cff7211d41f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:13.162406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b9d5c-6fd9-413d-9792-cff7211d41f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:13.164343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b9d5c-6fd9-413d-9792-cff7211d41f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:13.166074 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b9d5c-6fd9-413d-9792-cff7211d41f4 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:13.215540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 845/1685] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:13.217599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b9d5c-6fd9-413d-9792-cff7211d41f4 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:13.223940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ce0b9d5c-6fd9-413d-9792-cff7211d41f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:13.252995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b9d5c-6fd9-413d-9792-cff7211d41f4 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:13.264118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:13.267955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:13.268542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:13.269153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] PATH_INFO: `/v3/projects/f2bad68b6b024474b340ee4ce95b7156` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:13.271961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2bad68b6b024474b340ee4ce95b7156)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:13.280995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:13.288216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 841/1686] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:13 2022] POST /identity/v3/projects => generated 393 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:13.291716 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1204d4-64b0-4845-8a9b-08a2d4d9826e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:13.413568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1204d4-64b0-4845-8a9b-08a2d4d9826e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:13.425660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb193736-723c-4424-8bf7-5669f0dbd0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:13.427757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193736-723c-4424-8bf7-5669f0dbd0f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:13.428057 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193736-723c-4424-8bf7-5669f0dbd0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:13.428415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193736-723c-4424-8bf7-5669f0dbd0f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:13.429133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb193736-723c-4424-8bf7-5669f0dbd0f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:13.431062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb193736-723c-4424-8bf7-5669f0dbd0f7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:13.464754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2bad68b6b024474b340ee4ce95b7156', 'request_id': 'req-1d8db97a-8f24-4113-abce-c7dae61ba63e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:13.477481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2bad68b6b024474b340ee4ce95b7156', 'request_id': 'req-1d8db97a-8f24-4113-abce-c7dae61ba63e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:13.494082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2bad68b6b024474b340ee4ce95b7156', 'request_id': 'req-1d8db97a-8f24-4113-abce-c7dae61ba63e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:13.547161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 842/1687] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:13 2022] POST /identity/v3/projects => generated 365 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:13.548912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb193736-723c-4424-8bf7-5669f0dbd0f7 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:13.599661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] The token cache is being invalidate because project f2bad68b6b024474b340ee4ce95b7156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:13.599866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:13.600654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:13.606696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 846/1688] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:13 2022] DELETE /identity/v3/projects/f2bad68b6b024474b340ee4ce95b7156 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:13.606696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:13.609152 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:13.714658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8db97a-8f24-4113-abce-c7dae61ba63e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:13.730889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb193736-723c-4424-8bf7-5669f0dbd0f7 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:13.742100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba51029-ecec-424f-bb0e-fc726033f547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:13.744408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba51029-ecec-424f-bb0e-fc726033f547 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:13.746922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba51029-ecec-424f-bb0e-fc726033f547 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:13.747406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba51029-ecec-424f-bb0e-fc726033f547 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:13.748711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba51029-ecec-424f-bb0e-fc726033f547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:13.754986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba51029-ecec-424f-bb0e-fc726033f547 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:13.803055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13f4ed-c21a-40c2-99a2-e79d82e031b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:13.804581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13f4ed-c21a-40c2-99a2-e79d82e031b8 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:13.805339 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13f4ed-c21a-40c2-99a2-e79d82e031b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:13.805755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13f4ed-c21a-40c2-99a2-e79d82e031b8 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:13.808097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13f4ed-c21a-40c2-99a2-e79d82e031b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:13.810660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13f4ed-c21a-40c2-99a2-e79d82e031b8 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:13.889251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 843/1689] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:13 2022] POST /identity/v3/users => generated 297 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:13.892444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af3656-21f2-4e76-8118-259d7269464b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:13.892932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af3656-21f2-4e76-8118-259d7269464b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:13.894347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af3656-21f2-4e76-8118-259d7269464b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:13.936182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:13.945894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 847/1690] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:13 2022] POST /identity/v3/users => generated 283 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:13.947775 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13f4ed-c21a-40c2-99a2-e79d82e031b8 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:13.955779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-4b13f4ed-c21a-40c2-99a2-e79d82e031b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:13.962173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a9af3656-21f2-4e76-8118-259d7269464b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:13.968622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a9af3656-21f2-4e76-8118-259d7269464b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:13.989812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13f4ed-c21a-40c2-99a2-e79d82e031b8 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:13.990317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 844/1691] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:13.991221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af3656-21f2-4e76-8118-259d7269464b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:14.010697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af3656-21f2-4e76-8118-259d7269464b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:14.011561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af13c4f-0c74-49c5-986a-130707259728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:14.013388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af13c4f-0c74-49c5-986a-130707259728 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:14.013917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af13c4f-0c74-49c5-986a-130707259728 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:14.015253 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af13c4f-0c74-49c5-986a-130707259728 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:14.016153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af13c4f-0c74-49c5-986a-130707259728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:14.018726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:14.019541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f08192-8b38-4653-8361-cc6f5d02bfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:14.020289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f08192-8b38-4653-8361-cc6f5d02bfcf admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:14.020605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f08192-8b38-4653-8361-cc6f5d02bfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:14.021190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f08192-8b38-4653-8361-cc6f5d02bfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:14.021761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af13c4f-0c74-49c5-986a-130707259728 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:14.021932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f08192-8b38-4653-8361-cc6f5d02bfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:14.023734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f08192-8b38-4653-8361-cc6f5d02bfcf admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:14.037219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 848/1692] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:14.038628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 845/1693] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:14.038736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa198dd7-a9e4-4e27-8f75-450ec2033664 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:14.038953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa198dd7-a9e4-4e27-8f75-450ec2033664 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:14.039207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa198dd7-a9e4-4e27-8f75-450ec2033664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:14.039951 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f08192-8b38-4653-8361-cc6f5d02bfcf admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:14.053799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f08192-8b38-4653-8361-cc6f5d02bfcf admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:14.061089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311aa47-6a35-4941-a3e2-e1fe79099bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:14.061089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311aa47-6a35-4941-a3e2-e1fe79099bde admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:14.061089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311aa47-6a35-4941-a3e2-e1fe79099bde admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:14.061285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-aa198dd7-a9e4-4e27-8f75-450ec2033664 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:14.061383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311aa47-6a35-4941-a3e2-e1fe79099bde admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:14.061910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311aa47-6a35-4941-a3e2-e1fe79099bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:14.062191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-aa198dd7-a9e4-4e27-8f75-450ec2033664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:14.063656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311aa47-6a35-4941-a3e2-e1fe79099bde admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:14.078692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 846/1694] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:14.082563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 849/1695] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:14.082677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311aa47-6a35-4941-a3e2-e1fe79099bde admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:14.082765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa198dd7-a9e4-4e27-8f75-450ec2033664 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:14.095596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311aa47-6a35-4941-a3e2-e1fe79099bde admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:14.104904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa198dd7-a9e4-4e27-8f75-450ec2033664 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:14.109478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d59d3-7396-43e1-ad43-1771ee732ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:14.110183 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d59d3-7396-43e1-ad43-1771ee732ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:14.110486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d59d3-7396-43e1-ad43-1771ee732ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:14.112772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d59d3-7396-43e1-ad43-1771ee732ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:14.113486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d59d3-7396-43e1-ad43-1771ee732ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:14.128778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d59d3-7396-43e1-ad43-1771ee732ea3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:14.134616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6499c29-25c2-4a44-a187-098711fe3b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:14.135536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6499c29-25c2-4a44-a187-098711fe3b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:14.135826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6499c29-25c2-4a44-a187-098711fe3b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:14.136102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6499c29-25c2-4a44-a187-098711fe3b93 admin admin] PATH_INFO: `/v3/projects/733d9d99829b40a7b4cbad1e81b1dcd8/users/9e5b90b857674f3ba33e86448bf963e9/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:14.180450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 847/1696] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:14.181403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d59d3-7396-43e1-ad43-1771ee732ea3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:14.200047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6499c29-25c2-4a44-a187-098711fe3b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=733d9d99829b40a7b4cbad1e81b1dcd8, user_id=9e5b90b857674f3ba33e86448bf963e9, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:14.203526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6499c29-25c2-4a44-a187-098711fe3b93 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:14.221164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d59d3-7396-43e1-ad43-1771ee732ea3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:14.247626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c492a9-adaa-48dc-b722-5d9616b8b6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:14.247626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c492a9-adaa-48dc-b722-5d9616b8b6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:14.247626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c492a9-adaa-48dc-b722-5d9616b8b6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:14.247626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c492a9-adaa-48dc-b722-5d9616b8b6e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:14.247922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c492a9-adaa-48dc-b722-5d9616b8b6e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:14.247922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c492a9-adaa-48dc-b722-5d9616b8b6e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:14.256380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c492a9-adaa-48dc-b722-5d9616b8b6e7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:14.287398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 848/1697] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:14.288427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c492a9-adaa-48dc-b722-5d9616b8b6e7 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:14.307688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 850/1698] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:14 2022] PUT /identity/v3/projects/733d9d99829b40a7b4cbad1e81b1dcd8/users/9e5b90b857674f3ba33e86448bf963e9/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:14.318012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-81c492a9-adaa-48dc-b722-5d9616b8b6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:14.318192 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6499c29-25c2-4a44-a187-098711fe3b93 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:14.350871 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c492a9-adaa-48dc-b722-5d9616b8b6e7 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:14.560107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:14.561021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:14.561306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:14.561583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] PATH_INFO: `/v3/projects/8a024d1458954153b26bf70f811fa1b2` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:14.568714 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6499c29-25c2-4a44-a187-098711fe3b93 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:14.570957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a024d1458954153b26bf70f811fa1b2)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:14.573236 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:14.601094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f203aff-cbf1-472e-97e5-4270e3572488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:14.601769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f203aff-cbf1-472e-97e5-4270e3572488 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:14.602060 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f203aff-cbf1-472e-97e5-4270e3572488 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:14.602343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f203aff-cbf1-472e-97e5-4270e3572488 admin admin] PATH_INFO: `/v3/projects/0907ecc679014a37ac423ec2721c3178/users/c20e581aaca94614938550ade015418c/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:14.644397 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f203aff-cbf1-472e-97e5-4270e3572488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0907ecc679014a37ac423ec2721c3178, user_id=c20e581aaca94614938550ade015418c, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:14.647552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f203aff-cbf1-472e-97e5-4270e3572488 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:14.799622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 851/1699] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:14 2022] PUT /identity/v3/projects/0907ecc679014a37ac423ec2721c3178/users/c20e581aaca94614938550ade015418c/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:14.800835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f203aff-cbf1-472e-97e5-4270e3572488 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:14.811048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a024d1458954153b26bf70f811fa1b2', 'request_id': 'req-aa6a7d24-3500-4d1a-8727-b6d289e93e43'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:14.818786 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a024d1458954153b26bf70f811fa1b2', 'request_id': 'req-aa6a7d24-3500-4d1a-8727-b6d289e93e43'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:14.831595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a024d1458954153b26bf70f811fa1b2', 'request_id': 'req-aa6a7d24-3500-4d1a-8727-b6d289e93e43'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:14.912885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] The token cache is being invalidate because project 8a024d1458954153b26bf70f811fa1b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:14.913187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:14.915882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aa6a7d24-3500-4d1a-8727-b6d289e93e43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:14.925496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 849/1700] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:14 2022] DELETE /identity/v3/projects/8a024d1458954153b26bf70f811fa1b2 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:14.927015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b4ec5-a00a-495d-9404-a42ae346b401 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:14.929065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b4ec5-a00a-495d-9404-a42ae346b401 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:14.929505 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b4ec5-a00a-495d-9404-a42ae346b401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:14.984575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f203aff-cbf1-472e-97e5-4270e3572488 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:15.003046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a72b4ec5-a00a-495d-9404-a42ae346b401 None None] MFA Rules not processed for user `9e5b90b857674f3ba33e86448bf963e9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:15.004399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a72b4ec5-a00a-495d-9404-a42ae346b401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:15.105461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28339e7a-3d90-4952-90b0-6001ca9b2c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:15.106285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28339e7a-3d90-4952-90b0-6001ca9b2c87 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:15.107525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28339e7a-3d90-4952-90b0-6001ca9b2c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:15.107886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28339e7a-3d90-4952-90b0-6001ca9b2c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:15.108853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28339e7a-3d90-4952-90b0-6001ca9b2c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:15.111815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28339e7a-3d90-4952-90b0-6001ca9b2c87 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:15.169801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 850/1701] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:15.176178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a382331-b5ea-4249-b73c-8c856901bce7 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:15.176518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a382331-b5ea-4249-b73c-8c856901bce7 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:15.176811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a382331-b5ea-4249-b73c-8c856901bce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:15.208487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 852/1702] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:14 2022] POST /identity/v3/projects => generated 405 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:15.210769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8e8c5-a38d-4e3f-aad8-878a5acb3df0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:15.210887 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8e8c5-a38d-4e3f-aad8-878a5acb3df0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:15.211214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8e8c5-a38d-4e3f-aad8-878a5acb3df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:15.232523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-7a382331-b5ea-4249-b73c-8c856901bce7 None None] MFA Rules not processed for user `c20e581aaca94614938550ade015418c`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:15.234182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7a382331-b5ea-4249-b73c-8c856901bce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:15.244663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-35e8e8c5-a38d-4e3f-aad8-878a5acb3df0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:15.246596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-35e8e8c5-a38d-4e3f-aad8-878a5acb3df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:15.376489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 851/1703] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:15.377810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a382331-b5ea-4249-b73c-8c856901bce7 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:15.390278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7a382331-b5ea-4249-b73c-8c856901bce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:15.485086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 853/1704] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:15.490580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8e8c5-a38d-4e3f-aad8-878a5acb3df0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:15.498984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-35e8e8c5-a38d-4e3f-aad8-878a5acb3df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:15.549221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8e8c5-a38d-4e3f-aad8-878a5acb3df0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:15.573783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18372485-8a14-4c1e-a6cb-631fd3677e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:15.574430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18372485-8a14-4c1e-a6cb-631fd3677e40 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:15.577382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18372485-8a14-4c1e-a6cb-631fd3677e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:15.577382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18372485-8a14-4c1e-a6cb-631fd3677e40 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:15.577382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18372485-8a14-4c1e-a6cb-631fd3677e40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:15.592487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18372485-8a14-4c1e-a6cb-631fd3677e40 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:15.675041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a382331-b5ea-4249-b73c-8c856901bce7 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:15.687821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 854/1705] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:15 2022] POST /identity/v3/users => generated 303 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:15.693442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18372485-8a14-4c1e-a6cb-631fd3677e40 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:15.703514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-18372485-8a14-4c1e-a6cb-631fd3677e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:15.704286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caeb80e-13e2-487d-b712-1b5e2beca246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:15.705628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeb80e-13e2-487d-b712-1b5e2beca246 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:15.705919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeb80e-13e2-487d-b712-1b5e2beca246 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:15.706207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeb80e-13e2-487d-b712-1b5e2beca246 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:15.706921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caeb80e-13e2-487d-b712-1b5e2beca246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:15.709626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caeb80e-13e2-487d-b712-1b5e2beca246 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:15.820588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 852/1706] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:15 2022] POST /identity/v3/projects => generated 393 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:15.824393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880d27e-2924-401d-a2f1-6e5ac85a333d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:15.824720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880d27e-2924-401d-a2f1-6e5ac85a333d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:15.825011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880d27e-2924-401d-a2f1-6e5ac85a333d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:15.857729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b880d27e-2924-401d-a2f1-6e5ac85a333d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:15.859577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b880d27e-2924-401d-a2f1-6e5ac85a333d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:15.887240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18372485-8a14-4c1e-a6cb-631fd3677e40 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:15.999213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 853/1707] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:16.001267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880d27e-2924-401d-a2f1-6e5ac85a333d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:16.013311 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880d27e-2924-401d-a2f1-6e5ac85a333d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:16.018659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571276f-459f-428e-84bb-140c03efa9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:16.019352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571276f-459f-428e-84bb-140c03efa9ba admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.019641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571276f-459f-428e-84bb-140c03efa9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.019919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571276f-459f-428e-84bb-140c03efa9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.020728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571276f-459f-428e-84bb-140c03efa9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:16.022734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571276f-459f-428e-84bb-140c03efa9ba admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:16.044758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 854/1708] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:16.044758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571276f-459f-428e-84bb-140c03efa9ba admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:16.059441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571276f-459f-428e-84bb-140c03efa9ba admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:16.071462 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583ef24-ef11-4ec5-a08e-a026d77f21d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:16.072102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583ef24-ef11-4ec5-a08e-a026d77f21d8 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.072439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583ef24-ef11-4ec5-a08e-a026d77f21d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.072711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583ef24-ef11-4ec5-a08e-a026d77f21d8 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.075597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583ef24-ef11-4ec5-a08e-a026d77f21d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:16.083574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583ef24-ef11-4ec5-a08e-a026d77f21d8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:16.118844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935abffb-3b0c-4953-a6d8-c3ae21ecedca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:16.119400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935abffb-3b0c-4953-a6d8-c3ae21ecedca service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.119400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935abffb-3b0c-4953-a6d8-c3ae21ecedca service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.119685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935abffb-3b0c-4953-a6d8-c3ae21ecedca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.150111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:16.208083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 855/1709] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:16 2022] POST /identity/v3/users => generated 303 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:16.210249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583ef24-ef11-4ec5-a08e-a026d77f21d8 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:16.236307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583ef24-ef11-4ec5-a08e-a026d77f21d8 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:16.252186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e198c-d84a-4bdd-b40e-80e7a4a4fdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:16.254324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e198c-d84a-4bdd-b40e-80e7a4a4fdfd admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.254881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e198c-d84a-4bdd-b40e-80e7a4a4fdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.257010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e198c-d84a-4bdd-b40e-80e7a4a4fdfd admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.261544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e198c-d84a-4bdd-b40e-80e7a4a4fdfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:16.262259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e198c-d84a-4bdd-b40e-80e7a4a4fdfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:16.265586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e198c-d84a-4bdd-b40e-80e7a4a4fdfd admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:16.286007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:16.294280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935abffb-3b0c-4953-a6d8-c3ae21ecedca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:16.295871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935abffb-3b0c-4953-a6d8-c3ae21ecedca service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:16.303316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 856/1710] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:16.304275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e198c-d84a-4bdd-b40e-80e7a4a4fdfd admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:16.324637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 855/1711] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:16.329115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58d012-8134-4ed8-8193-cc1b8650f4d9 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.329319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58d012-8134-4ed8-8193-cc1b8650f4d9 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.331828 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58d012-8134-4ed8-8193-cc1b8650f4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.339381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e198c-d84a-4bdd-b40e-80e7a4a4fdfd admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:16.353642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870aed9-644b-41ad-8042-7bac77c32f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:16.358336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870aed9-644b-41ad-8042-7bac77c32f3b admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.358336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870aed9-644b-41ad-8042-7bac77c32f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.358336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870aed9-644b-41ad-8042-7bac77c32f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.358336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870aed9-644b-41ad-8042-7bac77c32f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:16.358336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870aed9-644b-41ad-8042-7bac77c32f3b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:16.369920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ae58d012-8134-4ed8-8193-cc1b8650f4d9 None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:16.372724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ae58d012-8134-4ed8-8193-cc1b8650f4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:16.395907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 857/1712] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:16.398514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870aed9-644b-41ad-8042-7bac77c32f3b admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:16.418901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870aed9-644b-41ad-8042-7bac77c32f3b admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:16.434038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6230ed-fa92-481f-84ad-09d004d9c293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:16.435783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6230ed-fa92-481f-84ad-09d004d9c293 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.435783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6230ed-fa92-481f-84ad-09d004d9c293 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.436127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6230ed-fa92-481f-84ad-09d004d9c293 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.437071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6230ed-fa92-481f-84ad-09d004d9c293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:16.449717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6230ed-fa92-481f-84ad-09d004d9c293 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:16.479502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 858/1713] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:16.481108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6230ed-fa92-481f-84ad-09d004d9c293 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:16.515559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6230ed-fa92-481f-84ad-09d004d9c293 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:16.531282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e243987-cea4-4459-83d3-cc8fd037651c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:16.531974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243987-cea4-4459-83d3-cc8fd037651c admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.532271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243987-cea4-4459-83d3-cc8fd037651c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.533879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 856/1714] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 15:56:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:16.534645 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243987-cea4-4459-83d3-cc8fd037651c admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.534916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58d012-8134-4ed8-8193-cc1b8650f4d9 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:16.537483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e243987-cea4-4459-83d3-cc8fd037651c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:16.549130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e243987-cea4-4459-83d3-cc8fd037651c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:16.555007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58d012-8134-4ed8-8193-cc1b8650f4d9 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:16.565579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535abb7-c4fa-44dc-803c-846042e042d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:16.566668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535abb7-c4fa-44dc-803c-846042e042d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.567080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535abb7-c4fa-44dc-803c-846042e042d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.567460 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535abb7-c4fa-44dc-803c-846042e042d6 admin admin] PATH_INFO: `/v3/projects/1d590cb287cd40bab547eefae785a7ce/users/391cb58098d94e07bee62fdeb306fd69/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.616606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535abb7-c4fa-44dc-803c-846042e042d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d590cb287cd40bab547eefae785a7ce, user_id=391cb58098d94e07bee62fdeb306fd69, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:16.620828 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535abb7-c4fa-44dc-803c-846042e042d6 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:16.679414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 859/1715] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:16 2022] POST /identity/v3/projects => generated 369 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:16.680634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e243987-cea4-4459-83d3-cc8fd037651c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:16.705413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 857/1716] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:16 2022] PUT /identity/v3/projects/1d590cb287cd40bab547eefae785a7ce/users/391cb58098d94e07bee62fdeb306fd69/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:16.707003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbc337-2a35-4d7f-9d7a-2c0609e854e4 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.707661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbc337-2a35-4d7f-9d7a-2c0609e854e4 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.708038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbc337-2a35-4d7f-9d7a-2c0609e854e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.748281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-41fbc337-2a35-4d7f-9d7a-2c0609e854e4 None None] MFA Rules not processed for user `83bb340d10e44bedad5fc9ddaf332552`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:16.750978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-41fbc337-2a35-4d7f-9d7a-2c0609e854e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:16.823823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e243987-cea4-4459-83d3-cc8fd037651c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:16.900552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 858/1717] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:16.902377 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbc337-2a35-4d7f-9d7a-2c0609e854e4 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:16.904779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b2eca-24b6-4b18-a39a-6e556340ccaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:16.905415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b2eca-24b6-4b18-a39a-6e556340ccaf admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.906154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b2eca-24b6-4b18-a39a-6e556340ccaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.906526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b2eca-24b6-4b18-a39a-6e556340ccaf admin admin] PATH_INFO: `/v3/projects/614c2054a8ce4250a944b3acf8857e32/users/b229b51093a748c0ad927d6cf0e821b2/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.920522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbc337-2a35-4d7f-9d7a-2c0609e854e4 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:16.927198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c7a66-c3d1-4d87-9bb4-e5ebad85e239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:16.928388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c7a66-c3d1-4d87-9bb4-e5ebad85e239 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:16.928825 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c7a66-c3d1-4d87-9bb4-e5ebad85e239 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:16.929156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c7a66-c3d1-4d87-9bb4-e5ebad85e239 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:16.929993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c7a66-c3d1-4d87-9bb4-e5ebad85e239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:16.930998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b2eca-24b6-4b18-a39a-6e556340ccaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=614c2054a8ce4250a944b3acf8857e32, user_id=b229b51093a748c0ad927d6cf0e821b2, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:16.931937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c7a66-c3d1-4d87-9bb4-e5ebad85e239 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:16.934298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b2eca-24b6-4b18-a39a-6e556340ccaf admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:17.033458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 860/1718] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:16 2022] PUT /identity/v3/projects/614c2054a8ce4250a944b3acf8857e32/users/b229b51093a748c0ad927d6cf0e821b2/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:17.034604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fc7ba-b74b-44e1-a121-602dbc75e724 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:17.034897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fc7ba-b74b-44e1-a121-602dbc75e724 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:17.035181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fc7ba-b74b-44e1-a121-602dbc75e724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:17.059060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a54fc7ba-b74b-44e1-a121-602dbc75e724 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:17.060279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a54fc7ba-b74b-44e1-a121-602dbc75e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:17.073795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 859/1719] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:16 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:17.074685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c7a66-c3d1-4d87-9bb4-e5ebad85e239 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:17.076803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7d1c7a66-c3d1-4d87-9bb4-e5ebad85e239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:17.175810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 861/1720] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:17.180922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fc7ba-b74b-44e1-a121-602dbc75e724 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:17.183034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a54fc7ba-b74b-44e1-a121-602dbc75e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:17.227162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c7a66-c3d1-4d87-9bb4-e5ebad85e239 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:17.309495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fc7ba-b74b-44e1-a121-602dbc75e724 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:17.318669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df1e82-a991-4341-9261-81192fd1711d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:17.319807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df1e82-a991-4341-9261-81192fd1711d service swift] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:17.320547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df1e82-a991-4341-9261-81192fd1711d service swift] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:17.321446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df1e82-a991-4341-9261-81192fd1711d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:17.401541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f037c8f-a7df-4806-8987-4ccbe59f7e7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:17.406358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f037c8f-a7df-4806-8987-4ccbe59f7e7a service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:17.408625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f037c8f-a7df-4806-8987-4ccbe59f7e7a service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:17.409017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f037c8f-a7df-4806-8987-4ccbe59f7e7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:17.424233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df1e82-a991-4341-9261-81192fd1711d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:17.427233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df1e82-a991-4341-9261-81192fd1711d service swift] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:17.440770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 862/1721] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 15:56:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:56:17.457877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df1e82-a991-4341-9261-81192fd1711d service swift] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:17.472014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df1e82-a991-4341-9261-81192fd1711d service swift] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:17.494509 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2c700-61e7-433e-9fe7-02ba1e9161a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:17.496517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2c700-61e7-433e-9fe7-02ba1e9161a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:17.497209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2c700-61e7-433e-9fe7-02ba1e9161a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:17.499998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2c700-61e7-433e-9fe7-02ba1e9161a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:17.522631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2c700-61e7-433e-9fe7-02ba1e9161a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:17.528931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2c700-61e7-433e-9fe7-02ba1e9161a4 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:17.574054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f037c8f-a7df-4806-8987-4ccbe59f7e7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:17.575747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f037c8f-a7df-4806-8987-4ccbe59f7e7a service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:17.583501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 863/1722] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:17.585573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2c700-61e7-433e-9fe7-02ba1e9161a4 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:17.605178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 860/1723] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:17.606137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e682a3-e62f-4f14-83d6-d21f55ffce68 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:17.606373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e682a3-e62f-4f14-83d6-d21f55ffce68 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:17.609521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e682a3-e62f-4f14-83d6-d21f55ffce68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:17.612546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2c700-61e7-433e-9fe7-02ba1e9161a4 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:17.630161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffb2f5-1cdf-4418-b017-062d1d260298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:17.631316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffb2f5-1cdf-4418-b017-062d1d260298 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:17.632018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffb2f5-1cdf-4418-b017-062d1d260298 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:17.632601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffb2f5-1cdf-4418-b017-062d1d260298 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:17.633498 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffb2f5-1cdf-4418-b017-062d1d260298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:17.638300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffb2f5-1cdf-4418-b017-062d1d260298 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:17.667710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:17.672817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 864/1724] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:17.676157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffb2f5-1cdf-4418-b017-062d1d260298 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:17.684184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-13e682a3-e62f-4f14-83d6-d21f55ffce68 None None] MFA Rules not processed for user `5b52924339de4cc08fe35dbf2a5fdcd0`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:17.686439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6fffb2f5-1cdf-4418-b017-062d1d260298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:17.687136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:17.689676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-13e682a3-e62f-4f14-83d6-d21f55ffce68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:17.821695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffb2f5-1cdf-4418-b017-062d1d260298 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:17.837460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a6ac1-adb6-435f-b965-5a4781af5d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:17.838424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a6ac1-adb6-435f-b965-5a4781af5d5f service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:17.838876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a6ac1-adb6-435f-b965-5a4781af5d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:17.839354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a6ac1-adb6-435f-b965-5a4781af5d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:17.864474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a6ac1-adb6-435f-b965-5a4781af5d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:17.868340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a6ac1-adb6-435f-b965-5a4781af5d5f service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:17.885705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 865/1725] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:17.886815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a6ac1-adb6-435f-b965-5a4781af5d5f service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:17.892749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-750a6ac1-adb6-435f-b965-5a4781af5d5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:17.917796 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a6ac1-adb6-435f-b965-5a4781af5d5f service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:17.930895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:17.931804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:17.932088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:17.932390 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] PATH_INFO: `/v3/users/3552d4b841ad47a2a89d3defa67e521b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:17.934167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3552d4b841ad47a2a89d3defa67e521b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:17.938025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:17.938477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 861/1726] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:17.939365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e682a3-e62f-4f14-83d6-d21f55ffce68 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:17.977695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e682a3-e62f-4f14-83d6-d21f55ffce68 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:17.996205 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0529a-8a74-40d1-9950-4a340e5d3977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:17.997850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0529a-8a74-40d1-9950-4a340e5d3977 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:17.998599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0529a-8a74-40d1-9950-4a340e5d3977 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:18.000194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0529a-8a74-40d1-9950-4a340e5d3977 admin admin] PATH_INFO: `/v3/projects/614c2054a8ce4250a944b3acf8857e32/users/b229b51093a748c0ad927d6cf0e821b2/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:18.007901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0529a-8a74-40d1-9950-4a340e5d3977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=614c2054a8ce4250a944b3acf8857e32, user_id=b229b51093a748c0ad927d6cf0e821b2, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:18.011869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0529a-8a74-40d1-9950-4a340e5d3977 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:18.067486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 862/1727] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:17 2022] PUT /identity/v3/projects/614c2054a8ce4250a944b3acf8857e32/users/b229b51093a748c0ad927d6cf0e821b2/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:18.067486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0529a-8a74-40d1-9950-4a340e5d3977 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:18.218145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3552d4b841ad47a2a89d3defa67e521b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:18.230915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3552d4b841ad47a2a89d3defa67e521b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:18.233694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3552d4b841ad47a2a89d3defa67e521b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:18.263046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3552d4b841ad47a2a89d3defa67e521b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:18.293659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0529a-8a74-40d1-9950-4a340e5d3977 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:18.318219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161ca6d-fe97-4afd-a866-8ae14e94c5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:18.318431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-35c03f40-5c83-4dea-94c5-08ae607e91e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3552d4b841ad47a2a89d3defa67e521b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:18.318945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161ca6d-fe97-4afd-a866-8ae14e94c5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:18.319180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161ca6d-fe97-4afd-a866-8ae14e94c5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:18.319458 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161ca6d-fe97-4afd-a866-8ae14e94c5a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:18.320191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161ca6d-fe97-4afd-a866-8ae14e94c5a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:18.325558 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161ca6d-fe97-4afd-a866-8ae14e94c5a3 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:18.358451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 863/1728] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:18.359661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a26e76-a96b-41cc-8150-8c51844c7ce0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:18.359959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a26e76-a96b-41cc-8150-8c51844c7ce0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:18.360218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a26e76-a96b-41cc-8150-8c51844c7ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:18.406821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-26a26e76-a96b-41cc-8150-8c51844c7ce0 None None] MFA Rules not processed for user `5b52924339de4cc08fe35dbf2a5fdcd0`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:18.409636 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-26a26e76-a96b-41cc-8150-8c51844c7ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:18.493939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:18.495795 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 866/1729] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:17 2022] DELETE /identity/v3/users/3552d4b841ad47a2a89d3defa67e521b => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:18.507872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f90da-53b0-4631-af8b-2fcbd798781b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:18.510421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f90da-53b0-4631-af8b-2fcbd798781b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:18.510781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f90da-53b0-4631-af8b-2fcbd798781b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:18.595809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:18.623753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e61f90da-53b0-4631-af8b-2fcbd798781b None None] MFA Rules not processed for user `b229b51093a748c0ad927d6cf0e821b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:18.625024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e61f90da-53b0-4631-af8b-2fcbd798781b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:18.641806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 864/1730] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:18.643417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a26e76-a96b-41cc-8150-8c51844c7ce0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:18.670635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-26a26e76-a96b-41cc-8150-8c51844c7ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:18.865581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a26e76-a96b-41cc-8150-8c51844c7ce0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:18.867329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 867/1731] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:18.867329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91f7e9-140f-4527-a598-01229b912dec None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:18.867329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91f7e9-140f-4527-a598-01229b912dec None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:18.867329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91f7e9-140f-4527-a598-01229b912dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:18.867329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0e91f7e9-140f-4527-a598-01229b912dec None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:18.867329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0e91f7e9-140f-4527-a598-01229b912dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:18.876829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420655e1-455f-486e-bd5b-986186280d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:18.878067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420655e1-455f-486e-bd5b-986186280d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:18.878355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420655e1-455f-486e-bd5b-986186280d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:18.878658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420655e1-455f-486e-bd5b-986186280d04 admin admin] PATH_INFO: `/v3/projects/faa6a40be0b6436a9453e8f981667eae/users/3e589c6dfc3a49c391bae7112946e64b/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:18.911886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420655e1-455f-486e-bd5b-986186280d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa6a40be0b6436a9453e8f981667eae, user_id=3e589c6dfc3a49c391bae7112946e64b, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:18.915837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420655e1-455f-486e-bd5b-986186280d04 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:18.973219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 868/1732] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:18.974452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91f7e9-140f-4527-a598-01229b912dec None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:18.988427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0e91f7e9-140f-4527-a598-01229b912dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:19.136224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91f7e9-140f-4527-a598-01229b912dec None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:19.145587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077934fb-3226-4b34-b370-50bf06c83bcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:19.146683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077934fb-3226-4b34-b370-50bf06c83bcc service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:19.146969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077934fb-3226-4b34-b370-50bf06c83bcc service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:19.147255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077934fb-3226-4b34-b370-50bf06c83bcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:19.149261 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-077934fb-3226-4b34-b370-50bf06c83bcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:19.253042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 865/1733] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:18 2022] PUT /identity/v3/projects/faa6a40be0b6436a9453e8f981667eae/users/3e589c6dfc3a49c391bae7112946e64b/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:19.266113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420655e1-455f-486e-bd5b-986186280d04 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:19.270557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-420655e1-455f-486e-bd5b-986186280d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:19.304763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077934fb-3226-4b34-b370-50bf06c83bcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:19.304763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077934fb-3226-4b34-b370-50bf06c83bcc service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:19.373640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 869/1734] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:19.374585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077934fb-3226-4b34-b370-50bf06c83bcc service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:19.377185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-077934fb-3226-4b34-b370-50bf06c83bcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:19.576387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420655e1-455f-486e-bd5b-986186280d04 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:19.586198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d13c8-937e-4456-a1d2-6d49ec5c96b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:19.586863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d13c8-937e-4456-a1d2-6d49ec5c96b6 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:19.587368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d13c8-937e-4456-a1d2-6d49ec5c96b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:19.587651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d13c8-937e-4456-a1d2-6d49ec5c96b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:19.666168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077934fb-3226-4b34-b370-50bf06c83bcc service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:19.848274 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d13c8-937e-4456-a1d2-6d49ec5c96b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:19.852522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d13c8-937e-4456-a1d2-6d49ec5c96b6 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:19.863097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98981b0b-6aef-470a-9996-7d695bcd02f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:19.864177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98981b0b-6aef-470a-9996-7d695bcd02f3 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:19.864651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98981b0b-6aef-470a-9996-7d695bcd02f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:19.865088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98981b0b-6aef-470a-9996-7d695bcd02f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:19.870528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 866/1735] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:19.872769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec1f9f-da90-4e9f-8c2f-d1d7e51a5d4d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:19.873119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec1f9f-da90-4e9f-8c2f-d1d7e51a5d4d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:19.873413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec1f9f-da90-4e9f-8c2f-d1d7e51a5d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:19.955286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98981b0b-6aef-470a-9996-7d695bcd02f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:19.956998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98981b0b-6aef-470a-9996-7d695bcd02f3 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:19.968801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 870/1736] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:19.970774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa9b1f-c63e-4256-9ec2-e01adfe25afe None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:19.971253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa9b1f-c63e-4256-9ec2-e01adfe25afe None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:19.971443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa9b1f-c63e-4256-9ec2-e01adfe25afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:19.983861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-58ec1f9f-da90-4e9f-8c2f-d1d7e51a5d4d None None] MFA Rules not processed for user `3e589c6dfc3a49c391bae7112946e64b`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:19.985362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-58ec1f9f-da90-4e9f-8c2f-d1d7e51a5d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:19.995624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-bcaa9b1f-c63e-4256-9ec2-e01adfe25afe None None] MFA Rules not processed for user `a39f5189d63d467dbd914f553c261c01`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:19.997959 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bcaa9b1f-c63e-4256-9ec2-e01adfe25afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:20.010907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 871/1737] 10.222.0.50 () {58 vars in 1015 bytes} [Mon May 23 15:56:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:20.028205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa9b1f-c63e-4256-9ec2-e01adfe25afe None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:20.031702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bcaa9b1f-c63e-4256-9ec2-e01adfe25afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:20.070766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:20.094165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa9b1f-c63e-4256-9ec2-e01adfe25afe None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:20.121229 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4f903-863f-4f44-8ffa-e3ffcec37780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:20.127494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4f903-863f-4f44-8ffa-e3ffcec37780 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:20.127494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4f903-863f-4f44-8ffa-e3ffcec37780 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:20.127494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4f903-863f-4f44-8ffa-e3ffcec37780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:20.127494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-21f4f903-863f-4f44-8ffa-e3ffcec37780 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:20.212422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 867/1738] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:19 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:20.213323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec1f9f-da90-4e9f-8c2f-d1d7e51a5d4d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:20.218923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-58ec1f9f-da90-4e9f-8c2f-d1d7e51a5d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:20.255397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4f903-863f-4f44-8ffa-e3ffcec37780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:20.258314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4f903-863f-4f44-8ffa-e3ffcec37780 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:20.272169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 872/1739] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:20.273330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4f903-863f-4f44-8ffa-e3ffcec37780 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:20.274240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec1f9f-da90-4e9f-8c2f-d1d7e51a5d4d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:20.288073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4f903-863f-4f44-8ffa-e3ffcec37780 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:20.293645 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c80bed-9ff4-4f57-a725-f66c04859472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:20.294573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c80bed-9ff4-4f57-a725-f66c04859472 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:20.294956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c80bed-9ff4-4f57-a725-f66c04859472 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:20.295373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c80bed-9ff4-4f57-a725-f66c04859472 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:20.295543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45812b-b096-4076-919a-1814f1e616e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:20.296219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45812b-b096-4076-919a-1814f1e616e6 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:20.296417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c80bed-9ff4-4f57-a725-f66c04859472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:20.296541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45812b-b096-4076-919a-1814f1e616e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:20.296945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45812b-b096-4076-919a-1814f1e616e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:20.300634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c80bed-9ff4-4f57-a725-f66c04859472 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:20.331044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45812b-b096-4076-919a-1814f1e616e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:20.332629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45812b-b096-4076-919a-1814f1e616e6 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:20.366852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 868/1740] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:20.432655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 873/1741] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:20 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:20.433431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45812b-b096-4076-919a-1814f1e616e6 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:20.451208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45812b-b096-4076-919a-1814f1e616e6 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:20.488232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:20.650891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b479ba-9a59-4e59-ac88-eabb150d3959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:20.652346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b479ba-9a59-4e59-ac88-eabb150d3959 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:20.656789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b479ba-9a59-4e59-ac88-eabb150d3959 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:20.661891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b479ba-9a59-4e59-ac88-eabb150d3959 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:20.661891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b479ba-9a59-4e59-ac88-eabb150d3959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:20.661891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b479ba-9a59-4e59-ac88-eabb150d3959 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:20.804733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 869/1742] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:20 2022] POST /identity/v3/users => generated 290 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:20.810089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c80bed-9ff4-4f57-a725-f66c04859472 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:20.860086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c80bed-9ff4-4f57-a725-f66c04859472 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:20.869946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3db66-13a7-4803-a298-0204ba807612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:20.870768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3db66-13a7-4803-a298-0204ba807612 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:20.871211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3db66-13a7-4803-a298-0204ba807612 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:20.871654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3db66-13a7-4803-a298-0204ba807612 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:20.873539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3db66-13a7-4803-a298-0204ba807612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:20.877027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3db66-13a7-4803-a298-0204ba807612 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:20.923686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 874/1743] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:20.924045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b479ba-9a59-4e59-ac88-eabb150d3959 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:20.938561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b479ba-9a59-4e59-ac88-eabb150d3959 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:20.950507 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136d117-93c8-4b1b-be96-6a2dc562776f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:20.953341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136d117-93c8-4b1b-be96-6a2dc562776f admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:20.953341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136d117-93c8-4b1b-be96-6a2dc562776f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:20.953341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136d117-93c8-4b1b-be96-6a2dc562776f admin admin] PATH_INFO: `/v3/projects/e0951a2277f547669cc89a0f63ae1b0c/users/8839c48f2d0146f4b693ec2fb0daea3b/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:20.976193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136d117-93c8-4b1b-be96-6a2dc562776f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0951a2277f547669cc89a0f63ae1b0c, user_id=8839c48f2d0146f4b693ec2fb0daea3b, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:20.982191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136d117-93c8-4b1b-be96-6a2dc562776f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:21.079351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 870/1744] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:20 2022] PUT /identity/v3/projects/e0951a2277f547669cc89a0f63ae1b0c/users/8839c48f2d0146f4b693ec2fb0daea3b/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:21.083750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3db66-13a7-4803-a298-0204ba807612 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:21.119977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3db66-13a7-4803-a298-0204ba807612 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:21.305195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be0531-fd9e-4d63-bc0d-5990510cefd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:21.307212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be0531-fd9e-4d63-bc0d-5990510cefd9 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:21.307510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be0531-fd9e-4d63-bc0d-5990510cefd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:21.313141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be0531-fd9e-4d63-bc0d-5990510cefd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:21.313141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be0531-fd9e-4d63-bc0d-5990510cefd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:21.315431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be0531-fd9e-4d63-bc0d-5990510cefd9 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:21.371357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 875/1745] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:21.374589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136d117-93c8-4b1b-be96-6a2dc562776f admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:21.396808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136d117-93c8-4b1b-be96-6a2dc562776f admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:21.403944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51943de7-e266-40e5-af94-156fa2752eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:21.406386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51943de7-e266-40e5-af94-156fa2752eda admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:21.408130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51943de7-e266-40e5-af94-156fa2752eda admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:21.408515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51943de7-e266-40e5-af94-156fa2752eda admin admin] PATH_INFO: `/v3/projects/e0951a2277f547669cc89a0f63ae1b0c/users/8839c48f2d0146f4b693ec2fb0daea3b/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:21.413869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51943de7-e266-40e5-af94-156fa2752eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0951a2277f547669cc89a0f63ae1b0c, user_id=8839c48f2d0146f4b693ec2fb0daea3b, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:21.421068 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51943de7-e266-40e5-af94-156fa2752eda admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:21.449561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 871/1746] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:21 2022] PUT /identity/v3/projects/e0951a2277f547669cc89a0f63ae1b0c/users/8839c48f2d0146f4b693ec2fb0daea3b/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:21.461513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427f8d4-d6c2-4c7b-836c-4c4ff124610b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:21.462021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427f8d4-d6c2-4c7b-836c-4c4ff124610b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:21.462094 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427f8d4-d6c2-4c7b-836c-4c4ff124610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:21.514274 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51943de7-e266-40e5-af94-156fa2752eda admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:21.516146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-51943de7-e266-40e5-af94-156fa2752eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:21.626680 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-c427f8d4-d6c2-4c7b-836c-4c4ff124610b None None] MFA Rules not processed for user `8839c48f2d0146f4b693ec2fb0daea3b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:21.636652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c427f8d4-d6c2-4c7b-836c-4c4ff124610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:21.648708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:21.742959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51943de7-e266-40e5-af94-156fa2752eda admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:21.759944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:21.761082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:21.761368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:21.761641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] PATH_INFO: `/v3/projects/fce9058356d34a8e875b5e166b4ac0a9` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:21.764494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fce9058356d34a8e875b5e166b4ac0a9)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:21.766779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:21.922275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 876/1747] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:21 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:21.935012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427f8d4-d6c2-4c7b-836c-4c4ff124610b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:22.009860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fce9058356d34a8e875b5e166b4ac0a9', 'request_id': 'req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:22.024662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fce9058356d34a8e875b5e166b4ac0a9', 'request_id': 'req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:22.035650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fce9058356d34a8e875b5e166b4ac0a9', 'request_id': 'req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:22.077007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427f8d4-d6c2-4c7b-836c-4c4ff124610b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:22.126671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] The token cache is being invalidate because project fce9058356d34a8e875b5e166b4ac0a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:22.126974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:22.137082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-11b1e5b1-a4cc-42f1-bc6d-d5d0cce0c18c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:22.140861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 872/1748] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:21 2022] DELETE /identity/v3/projects/fce9058356d34a8e875b5e166b4ac0a9 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:22.152236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b9907-3e02-4c49-a456-11d5425d1d21 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:22.152559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b9907-3e02-4c49-a456-11d5425d1d21 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:22.152849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b9907-3e02-4c49-a456-11d5425d1d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:22.214005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a69b9907-3e02-4c49-a456-11d5425d1d21 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:22.215080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a69b9907-3e02-4c49-a456-11d5425d1d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:22.302651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29495f6-d121-4a30-a8b9-29f10833841f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:22.303576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29495f6-d121-4a30-a8b9-29f10833841f service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:22.305875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29495f6-d121-4a30-a8b9-29f10833841f service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:22.306626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29495f6-d121-4a30-a8b9-29f10833841f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:22.314758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e29495f6-d121-4a30-a8b9-29f10833841f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:22.416815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 873/1749] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:22.425370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9b25e-8697-4291-aaa6-2183533c60b6 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:22.425666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9b25e-8697-4291-aaa6-2183533c60b6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:22.425965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9b25e-8697-4291-aaa6-2183533c60b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:22.471401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29495f6-d121-4a30-a8b9-29f10833841f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:22.473260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29495f6-d121-4a30-a8b9-29f10833841f service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:22.504719 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 877/1750] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:21 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:56:22.505760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-dfb9b25e-8697-4291-aaa6-2183533c60b6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:22.507146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dfb9b25e-8697-4291-aaa6-2183533c60b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:22.546712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 874/1751] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:22.548117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29495f6-d121-4a30-a8b9-29f10833841f service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:22.563360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29495f6-d121-4a30-a8b9-29f10833841f service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:22.583652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ddc7-ef75-4c4c-be16-96c10895dcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:22.586345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ddc7-ef75-4c4c-be16-96c10895dcf6 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:22.587876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ddc7-ef75-4c4c-be16-96c10895dcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:22.587876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13ddc7-ef75-4c4c-be16-96c10895dcf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:22.590963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13ddc7-ef75-4c4c-be16-96c10895dcf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:22.592186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13ddc7-ef75-4c4c-be16-96c10895dcf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:22.597265 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df13ddc7-ef75-4c4c-be16-96c10895dcf6 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:22.635927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:22.664639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 878/1752] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:22.668523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9b25e-8697-4291-aaa6-2183533c60b6 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:22.689560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9b25e-8697-4291-aaa6-2183533c60b6 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:22.692512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ddc7-ef75-4c4c-be16-96c10895dcf6 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:22.696784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-df13ddc7-ef75-4c4c-be16-96c10895dcf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:22.721921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971046c-25da-41ee-97bd-539dd5efe358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:22.724079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971046c-25da-41ee-97bd-539dd5efe358 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:22.726691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971046c-25da-41ee-97bd-539dd5efe358 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:22.727182 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971046c-25da-41ee-97bd-539dd5efe358 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:22.728115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971046c-25da-41ee-97bd-539dd5efe358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:22.730319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971046c-25da-41ee-97bd-539dd5efe358 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:22.744247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13ddc7-ef75-4c4c-be16-96c10895dcf6 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:22.768040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93d024-20ed-49d5-b06b-133fc87a380e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:22.768924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93d024-20ed-49d5-b06b-133fc87a380e service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:22.769313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93d024-20ed-49d5-b06b-133fc87a380e service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:22.769834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93d024-20ed-49d5-b06b-133fc87a380e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:22.796981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5f93d024-20ed-49d5-b06b-133fc87a380e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:22.927833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93d024-20ed-49d5-b06b-133fc87a380e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:22.932642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93d024-20ed-49d5-b06b-133fc87a380e service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:22.948131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 875/1753] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:22 2022] POST /identity/v3/projects => generated 363 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:56:22.949304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971046c-25da-41ee-97bd-539dd5efe358 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:22.960127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 879/1754] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:23.238011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971046c-25da-41ee-97bd-539dd5efe358 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:23.252691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6cc3f-3d00-46c9-8564-1d255521b276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:23.253406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6cc3f-3d00-46c9-8564-1d255521b276 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:23.253692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6cc3f-3d00-46c9-8564-1d255521b276 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:23.253976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6cc3f-3d00-46c9-8564-1d255521b276 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:23.254679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6cc3f-3d00-46c9-8564-1d255521b276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:23.257033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6cc3f-3d00-46c9-8564-1d255521b276 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:23.420315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 876/1755] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:22 2022] POST /identity/v3/users => generated 282 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:23.424165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93d024-20ed-49d5-b06b-133fc87a380e service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:23.442938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93d024-20ed-49d5-b06b-133fc87a380e service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:23.475807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d429eeb-9cb9-4034-a2dc-bfe01ef379fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:23.479973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d429eeb-9cb9-4034-a2dc-bfe01ef379fa admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:23.481700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d429eeb-9cb9-4034-a2dc-bfe01ef379fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:23.481700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d429eeb-9cb9-4034-a2dc-bfe01ef379fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:23.486300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d429eeb-9cb9-4034-a2dc-bfe01ef379fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:23.509259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d429eeb-9cb9-4034-a2dc-bfe01ef379fa admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:23.590477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:23.592248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 880/1756] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:23.596822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6cc3f-3d00-46c9-8564-1d255521b276 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:23.612154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6cc3f-3d00-46c9-8564-1d255521b276 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:23.624014 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381d029-03d5-43f8-adca-fff00b620ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:23.624929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381d029-03d5-43f8-adca-fff00b620ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:23.625323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381d029-03d5-43f8-adca-fff00b620ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:23.633004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381d029-03d5-43f8-adca-fff00b620ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:23.633429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381d029-03d5-43f8-adca-fff00b620ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:23.637225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381d029-03d5-43f8-adca-fff00b620ab0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:23.680666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 877/1757] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:23.681102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d429eeb-9cb9-4034-a2dc-bfe01ef379fa admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:23.695117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d429eeb-9cb9-4034-a2dc-bfe01ef379fa admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:23.717050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d629b06-27d0-4780-8ebd-9e91dea8beae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:23.717976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d629b06-27d0-4780-8ebd-9e91dea8beae admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:23.718263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d629b06-27d0-4780-8ebd-9e91dea8beae admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:23.718545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d629b06-27d0-4780-8ebd-9e91dea8beae admin admin] PATH_INFO: `/v3/projects/b9d76514996e43a8a6f90e658dd4bced/users/65ebeb2243fe4a4781b6d25880bd8c79/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:23.774248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d629b06-27d0-4780-8ebd-9e91dea8beae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d76514996e43a8a6f90e658dd4bced, user_id=65ebeb2243fe4a4781b6d25880bd8c79, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:23.775297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d629b06-27d0-4780-8ebd-9e91dea8beae admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:23.872017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac1fc1-b37b-4ca5-9f25-8d16354b4a5c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:23.872721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac1fc1-b37b-4ca5-9f25-8d16354b4a5c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:23.873173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac1fc1-b37b-4ca5-9f25-8d16354b4a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:23.931630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2aac1fc1-b37b-4ca5-9f25-8d16354b4a5c None None] MFA Rules not processed for user `5b52924339de4cc08fe35dbf2a5fdcd0`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:23.931630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2aac1fc1-b37b-4ca5-9f25-8d16354b4a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:23.947371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 881/1758] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:23 2022] PUT /identity/v3/projects/b9d76514996e43a8a6f90e658dd4bced/users/65ebeb2243fe4a4781b6d25880bd8c79/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:23.951638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f624c9d-bc93-4ccf-8d99-e7c57c649881 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:23.952132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f624c9d-bc93-4ccf-8d99-e7c57c649881 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:23.953208 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f624c9d-bc93-4ccf-8d99-e7c57c649881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:24.026365 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-9f624c9d-bc93-4ccf-8d99-e7c57c649881 None None] MFA Rules not processed for user `65ebeb2243fe4a4781b6d25880bd8c79`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:24.027792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-9f624c9d-bc93-4ccf-8d99-e7c57c649881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:24.112023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 878/1759] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:24.113422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a41bd-8b1d-4845-b8d6-f82d7f90f390 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:24.113711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a41bd-8b1d-4845-b8d6-f82d7f90f390 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:24.113994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a41bd-8b1d-4845-b8d6-f82d7f90f390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:24.237797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 882/1760] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:23 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:24.238772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f624c9d-bc93-4ccf-8d99-e7c57c649881 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:24.251603 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-7b0a41bd-8b1d-4845-b8d6-f82d7f90f390 None None] MFA Rules not processed for user `391cb58098d94e07bee62fdeb306fd69`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:24.253846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7b0a41bd-8b1d-4845-b8d6-f82d7f90f390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:24.464450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f624c9d-bc93-4ccf-8d99-e7c57c649881 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:24.484239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db991167-6233-4071-8e36-3661bcbe0d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:24.513461 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db991167-6233-4071-8e36-3661bcbe0d43 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:24.514779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db991167-6233-4071-8e36-3661bcbe0d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:24.515340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db991167-6233-4071-8e36-3661bcbe0d43 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:24.517729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db991167-6233-4071-8e36-3661bcbe0d43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:24.522931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db991167-6233-4071-8e36-3661bcbe0d43 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:24.536918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 879/1761] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:24 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:24.540822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113ba63-d995-4819-9447-5c1d351b7bce None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:24.541544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113ba63-d995-4819-9447-5c1d351b7bce None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:24.542280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113ba63-d995-4819-9447-5c1d351b7bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:24.611385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e113ba63-d995-4819-9447-5c1d351b7bce None None] MFA Rules not processed for user `83bb340d10e44bedad5fc9ddaf332552`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:24.612722 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e113ba63-d995-4819-9447-5c1d351b7bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:24.709790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 883/1762] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:24 2022] POST /identity/v3/projects => generated 365 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:24.709790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db991167-6233-4071-8e36-3661bcbe0d43 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:24.713477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-db991167-6233-4071-8e36-3661bcbe0d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:24.713565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 880/1763] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:24.713565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113ba63-d995-4819-9447-5c1d351b7bce None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:24.753257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113ba63-d995-4819-9447-5c1d351b7bce None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:24.824521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:24.937943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db991167-6233-4071-8e36-3661bcbe0d43 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:24.946479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40099beb-f068-4796-9641-19fb7c990486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:24.947824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099beb-f068-4796-9641-19fb7c990486 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:24.948375 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099beb-f068-4796-9641-19fb7c990486 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:24.948914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099beb-f068-4796-9641-19fb7c990486 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:24.950049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40099beb-f068-4796-9641-19fb7c990486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:24.952880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40099beb-f068-4796-9641-19fb7c990486 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:24.971054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d714b1-cf90-4951-8a21-4c67a89ed785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:24.971818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d714b1-cf90-4951-8a21-4c67a89ed785 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:24.972097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d714b1-cf90-4951-8a21-4c67a89ed785 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:24.972403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d714b1-cf90-4951-8a21-4c67a89ed785 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:24.973270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d714b1-cf90-4951-8a21-4c67a89ed785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:24.977436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d714b1-cf90-4951-8a21-4c67a89ed785 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:25.075304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 884/1764] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:24 2022] POST /identity/v3/projects => generated 403 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:25.076890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298ee7b-beba-40a7-b02f-d567af864dcc None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:25.077187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298ee7b-beba-40a7-b02f-d567af864dcc None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:25.077468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298ee7b-beba-40a7-b02f-d567af864dcc None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:25.078829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 885/1765] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 15:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:56:25.083996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298ee7b-beba-40a7-b02f-d567af864dcc None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:25.112572 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 881/1766] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:24 2022] POST /identity/v3/users => generated 289 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:25.114426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a46f8-601b-4faf-9c5e-ecb71d96dbe9 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:25.114883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a46f8-601b-4faf-9c5e-ecb71d96dbe9 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:25.115348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a46f8-601b-4faf-9c5e-ecb71d96dbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:25.154004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c55a46f8-601b-4faf-9c5e-ecb71d96dbe9 None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:25.157687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c55a46f8-601b-4faf-9c5e-ecb71d96dbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:25.240501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 882/1767] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 15:56:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:25.241577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a46f8-601b-4faf-9c5e-ecb71d96dbe9 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:25.242622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298ee7b-beba-40a7-b02f-d567af864dcc None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:25.253386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a46f8-601b-4faf-9c5e-ecb71d96dbe9 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:25.259247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0373d9e-7265-42e2-a662-68b010c3fbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:25.261061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e5f05-feee-4288-96b4-469be2307c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:25.261395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0373d9e-7265-42e2-a662-68b010c3fbfa admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:25.261774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e5f05-feee-4288-96b4-469be2307c4b admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:25.262064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e5f05-feee-4288-96b4-469be2307c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:25.262346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e5f05-feee-4288-96b4-469be2307c4b admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:25.262782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0373d9e-7265-42e2-a662-68b010c3fbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:25.266338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e5f05-feee-4288-96b4-469be2307c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:25.274860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e5f05-feee-4288-96b4-469be2307c4b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:25.278257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0373d9e-7265-42e2-a662-68b010c3fbfa admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:25.279344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0373d9e-7265-42e2-a662-68b010c3fbfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:25.283237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0373d9e-7265-42e2-a662-68b010c3fbfa admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:25.313816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 883/1768] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:25.318163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0373d9e-7265-42e2-a662-68b010c3fbfa admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:25.339045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0373d9e-7265-42e2-a662-68b010c3fbfa admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:25.345090 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208e69e-c206-4230-9e60-e9afb4f64ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:25.347159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208e69e-c206-4230-9e60-e9afb4f64ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:25.347507 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208e69e-c206-4230-9e60-e9afb4f64ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:25.347925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208e69e-c206-4230-9e60-e9afb4f64ef1 admin admin] PATH_INFO: `/v3/projects/407115775c204537b78db32087198814/users/580eaaaa79894a058ef2f0fc35466d0f/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:25.411377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 886/1769] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:25 2022] POST /identity/v3/users => generated 308 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:25.414768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e5f05-feee-4288-96b4-469be2307c4b admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:25.416198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208e69e-c206-4230-9e60-e9afb4f64ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=407115775c204537b78db32087198814, user_id=580eaaaa79894a058ef2f0fc35466d0f, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:25.431667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208e69e-c206-4230-9e60-e9afb4f64ef1 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:25.449631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e5f05-feee-4288-96b4-469be2307c4b admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:25.456672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ec19a-a3bb-41a7-ac5b-427ed6acb419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:25.457324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ec19a-a3bb-41a7-ac5b-427ed6acb419 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:25.457618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ec19a-a3bb-41a7-ac5b-427ed6acb419 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:25.457904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ec19a-a3bb-41a7-ac5b-427ed6acb419 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:25.458780 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ec19a-a3bb-41a7-ac5b-427ed6acb419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:25.462278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ec19a-a3bb-41a7-ac5b-427ed6acb419 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:25.478114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 887/1770] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:25.487346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ec19a-a3bb-41a7-ac5b-427ed6acb419 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:25.509760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ec19a-a3bb-41a7-ac5b-427ed6acb419 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:25.535661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf503b1-9ae9-44be-bd03-a068f0ca7e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:25.536618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf503b1-9ae9-44be-bd03-a068f0ca7e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:25.536911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf503b1-9ae9-44be-bd03-a068f0ca7e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:25.537189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf503b1-9ae9-44be-bd03-a068f0ca7e41 admin admin] PATH_INFO: `/v3/projects/358f5c0b2a374a1eaa7b36041f48c765/users/400cd6b39a234c5885f6262075548f48/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:25.574483 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 884/1771] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:25 2022] PUT /identity/v3/projects/407115775c204537b78db32087198814/users/580eaaaa79894a058ef2f0fc35466d0f/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:25.578862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208e69e-c206-4230-9e60-e9afb4f64ef1 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:25.636846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf503b1-9ae9-44be-bd03-a068f0ca7e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=358f5c0b2a374a1eaa7b36041f48c765, user_id=400cd6b39a234c5885f6262075548f48, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:25.649426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf503b1-9ae9-44be-bd03-a068f0ca7e41 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:25.804530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 888/1772] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:25 2022] PUT /identity/v3/projects/358f5c0b2a374a1eaa7b36041f48c765/users/400cd6b39a234c5885f6262075548f48/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:25.805992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaff6e8-fbc5-4e83-9f64-4953d4750f4c None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:25.806290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaff6e8-fbc5-4e83-9f64-4953d4750f4c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:25.806618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaff6e8-fbc5-4e83-9f64-4953d4750f4c None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:25.808374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 889/1773] 10.222.0.23 () {54 vars in 912 bytes} [Mon May 23 15:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:56:25.809112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaff6e8-fbc5-4e83-9f64-4953d4750f4c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:25.825843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208e69e-c206-4230-9e60-e9afb4f64ef1 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:25.992433 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaff6e8-fbc5-4e83-9f64-4953d4750f4c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:26.021121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76a459-55af-4592-980a-3f71b1453342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:26.022056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76a459-55af-4592-980a-3f71b1453342 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:26.022337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76a459-55af-4592-980a-3f71b1453342 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:26.023394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457819a5-c096-493a-8b59-b43cece9246f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:26.024125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457819a5-c096-493a-8b59-b43cece9246f admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:26.024346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457819a5-c096-493a-8b59-b43cece9246f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:26.027111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457819a5-c096-493a-8b59-b43cece9246f admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:26.027330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76a459-55af-4592-980a-3f71b1453342 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:26.029294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457819a5-c096-493a-8b59-b43cece9246f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:26.030638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76a459-55af-4592-980a-3f71b1453342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:26.034626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76a459-55af-4592-980a-3f71b1453342 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:26.039494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457819a5-c096-493a-8b59-b43cece9246f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:26.068310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 885/1774] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:26.069376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 890/1775] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:26.071646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb165f-e111-40b6-8835-533696d5c16d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:26.072003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76a459-55af-4592-980a-3f71b1453342 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:26.072926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb165f-e111-40b6-8835-533696d5c16d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:26.078556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb165f-e111-40b6-8835-533696d5c16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:26.101689 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76a459-55af-4592-980a-3f71b1453342 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:26.114255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e35a2e-a170-4df7-a4b4-fce5952e1dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:26.115366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e35a2e-a170-4df7-a4b4-fce5952e1dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:26.115652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e35a2e-a170-4df7-a4b4-fce5952e1dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:26.115934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e35a2e-a170-4df7-a4b4-fce5952e1dcb admin admin] PATH_INFO: `/v3/projects/407115775c204537b78db32087198814/users/580eaaaa79894a058ef2f0fc35466d0f/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:26.121055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e35a2e-a170-4df7-a4b4-fce5952e1dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=407115775c204537b78db32087198814, user_id=580eaaaa79894a058ef2f0fc35466d0f, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:26.129087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e35a2e-a170-4df7-a4b4-fce5952e1dcb admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:26.130698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8ebb165f-e111-40b6-8835-533696d5c16d None None] MFA Rules not processed for user `a39f5189d63d467dbd914f553c261c01`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:26.141246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8ebb165f-e111-40b6-8835-533696d5c16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:26.162257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:26.175276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 891/1776] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:26 2022] PUT /identity/v3/projects/407115775c204537b78db32087198814/users/580eaaaa79894a058ef2f0fc35466d0f/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:26.175276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e35a2e-a170-4df7-a4b4-fce5952e1dcb admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:26.278921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 886/1777] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 15:56:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:26.280286 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3176d1-6db3-48ce-b4fb-fbe611685359 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:26.280603 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3176d1-6db3-48ce-b4fb-fbe611685359 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:26.280873 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3176d1-6db3-48ce-b4fb-fbe611685359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:26.340751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e35a2e-a170-4df7-a4b4-fce5952e1dcb admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:26.352964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b8ed3-d36e-4362-a680-1c9cecec005c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:26.362699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-5e3176d1-6db3-48ce-b4fb-fbe611685359 None None] MFA Rules not processed for user `580eaaaa79894a058ef2f0fc35466d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:26.364050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5e3176d1-6db3-48ce-b4fb-fbe611685359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:26.368686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b8ed3-d36e-4362-a680-1c9cecec005c admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:26.370648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b8ed3-d36e-4362-a680-1c9cecec005c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:26.371336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b8ed3-d36e-4362-a680-1c9cecec005c admin admin] PATH_INFO: `/v3/projects/358f5c0b2a374a1eaa7b36041f48c765/users/400cd6b39a234c5885f6262075548f48/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:26.378037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b8ed3-d36e-4362-a680-1c9cecec005c admin admin] RBAC: Authorizing `identity:create_grant(project_id=358f5c0b2a374a1eaa7b36041f48c765, user_id=400cd6b39a234c5885f6262075548f48, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:26.387025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b8ed3-d36e-4362-a680-1c9cecec005c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:26.415683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 892/1778] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:26 2022] PUT /identity/v3/projects/358f5c0b2a374a1eaa7b36041f48c765/users/400cd6b39a234c5885f6262075548f48/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:26.416675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b8ed3-d36e-4362-a680-1c9cecec005c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:26.433319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-491b8ed3-d36e-4362-a680-1c9cecec005c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:26.497109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 887/1779] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:26.510043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3176d1-6db3-48ce-b4fb-fbe611685359 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:26.597295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3176d1-6db3-48ce-b4fb-fbe611685359 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:26.603228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b8ed3-d36e-4362-a680-1c9cecec005c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:26.612170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cac0a-fd5b-4ab5-b3aa-a3ac411cc6fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:26.612881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cac0a-fd5b-4ab5-b3aa-a3ac411cc6fe service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:26.613168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cac0a-fd5b-4ab5-b3aa-a3ac411cc6fe service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:26.613454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cac0a-fd5b-4ab5-b3aa-a3ac411cc6fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:26.634511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cac0a-fd5b-4ab5-b3aa-a3ac411cc6fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:26.636015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cac0a-fd5b-4ab5-b3aa-a3ac411cc6fe service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:26.670642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 893/1780] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:26.720054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0aaae-7332-4956-9f5e-f3f06690fcf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:26.721334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0aaae-7332-4956-9f5e-f3f06690fcf5 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:26.721619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0aaae-7332-4956-9f5e-f3f06690fcf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:26.721894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0aaae-7332-4956-9f5e-f3f06690fcf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:26.831565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0aaae-7332-4956-9f5e-f3f06690fcf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:26.833426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0aaae-7332-4956-9f5e-f3f06690fcf5 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:26.868171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 888/1781] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:27.065445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:27.870868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cac0a-fd5b-4ab5-b3aa-a3ac411cc6fe service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:27.872339 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-651cac0a-fd5b-4ab5-b3aa-a3ac411cc6fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:27.941439 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0aaae-7332-4956-9f5e-f3f06690fcf5 service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:27.943801 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dcc0aaae-7332-4956-9f5e-f3f06690fcf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:27.957228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:28.062601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cac0a-fd5b-4ab5-b3aa-a3ac411cc6fe service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:28.077233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ef065-0497-43c6-9baa-57c50deceaaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:28.078155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ef065-0497-43c6-9baa-57c50deceaaf service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:28.078447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ef065-0497-43c6-9baa-57c50deceaaf service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:28.079192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ef065-0497-43c6-9baa-57c50deceaaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:28.094216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ef065-0497-43c6-9baa-57c50deceaaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:28.096770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ef065-0497-43c6-9baa-57c50deceaaf service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:28.135029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 894/1782] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:28.260477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0aaae-7332-4956-9f5e-f3f06690fcf5 service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:28.304914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:28.309224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:28.316488 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:28.316795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] PATH_INFO: `/v3/users/9e5b90b857674f3ba33e86448bf963e9` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:28.334067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5b90b857674f3ba33e86448bf963e9)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:28.343244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:28.757366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5b90b857674f3ba33e86448bf963e9'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:28.766267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5b90b857674f3ba33e86448bf963e9'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:28.784195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5b90b857674f3ba33e86448bf963e9'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:28.786133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5b90b857674f3ba33e86448bf963e9'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:28.796167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-69a5764b-4def-4617-bff9-312ea5a57e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5b90b857674f3ba33e86448bf963e9'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:28.844167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ef065-0497-43c6-9baa-57c50deceaaf service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:28.859317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ef065-0497-43c6-9baa-57c50deceaaf service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:28.869446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:28.871435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:28.872601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:28.873359 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] PATH_INFO: `/v3/users/65ebeb2243fe4a4781b6d25880bd8c79` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:28.875951 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ebeb2243fe4a4781b6d25880bd8c79)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:28.879831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:28.898918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 889/1783] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:27 2022] DELETE /identity/v3/users/9e5b90b857674f3ba33e86448bf963e9 => generated 0 bytes in 958 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:28.900274 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d8e92-4b46-473e-9a9a-05f6f9ecacad None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:28.900595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d8e92-4b46-473e-9a9a-05f6f9ecacad None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:28.900879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d8e92-4b46-473e-9a9a-05f6f9ecacad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:28.964914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6c7d8e92-4b46-473e-9a9a-05f6f9ecacad None None] MFA Rules not processed for user `400cd6b39a234c5885f6262075548f48`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:28.966728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6c7d8e92-4b46-473e-9a9a-05f6f9ecacad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:29.110861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:29.131046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 890/1784] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:28 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:29.134911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93d97b-4584-4169-a9c8-677ba6a2b71e None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:29.135523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93d97b-4584-4169-a9c8-677ba6a2b71e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:29.135991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93d97b-4584-4169-a9c8-677ba6a2b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:29.161937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8c93d97b-4584-4169-a9c8-677ba6a2b71e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:29.163757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8c93d97b-4584-4169-a9c8-677ba6a2b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:29.224860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ebeb2243fe4a4781b6d25880bd8c79'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:29.225580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ebeb2243fe4a4781b6d25880bd8c79'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:29.226315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ebeb2243fe4a4781b6d25880bd8c79'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:29.250774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ebeb2243fe4a4781b6d25880bd8c79'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:29.283977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ebeb2243fe4a4781b6d25880bd8c79'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:29.304848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 891/1785] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:29.373149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 895/1786] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:28 2022] DELETE /identity/v3/users/65ebeb2243fe4a4781b6d25880bd8c79 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:29.376857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eef0d0-069c-44ea-806a-9fb57cb0da8f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:29.377423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eef0d0-069c-44ea-806a-9fb57cb0da8f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:29.377891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eef0d0-069c-44ea-806a-9fb57cb0da8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:29.431381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:29.433496 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:29.441337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-12eef0d0-069c-44ea-806a-9fb57cb0da8f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:29.442616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-12eef0d0-069c-44ea-806a-9fb57cb0da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:29.672629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edf847-d0d0-4ce2-9013-829878ceadb1 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:29.685315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6323cd-dd12-498d-a930-8d1e1b816c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:29.685992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6323cd-dd12-498d-a930-8d1e1b816c83 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:29.686275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6323cd-dd12-498d-a930-8d1e1b816c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:29.686588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6323cd-dd12-498d-a930-8d1e1b816c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:29.738065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 892/1787] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:29.742254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:29.771950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eef0d0-069c-44ea-806a-9fb57cb0da8f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:29.773780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-12eef0d0-069c-44ea-806a-9fb57cb0da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:29.799071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eef0d0-069c-44ea-806a-9fb57cb0da8f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:29.803460 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5d1f6-6e5a-4a4b-b5e1-526fafced8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:29.804434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5d1f6-6e5a-4a4b-b5e1-526fafced8af service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:29.804929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5d1f6-6e5a-4a4b-b5e1-526fafced8af service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:29.805460 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5d1f6-6e5a-4a4b-b5e1-526fafced8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:29.829463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5d1f6-6e5a-4a4b-b5e1-526fafced8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:29.831827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5d1f6-6e5a-4a4b-b5e1-526fafced8af service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:29.859716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 893/1788] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:29.891157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6323cd-dd12-498d-a930-8d1e1b816c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:29.894530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6323cd-dd12-498d-a930-8d1e1b816c83 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:29.933450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 896/1789] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:56:29.969724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:30.492538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5d1f6-6e5a-4a4b-b5e1-526fafced8af service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:30.495234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-42c5d1f6-6e5a-4a4b-b5e1-526fafced8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:30.524781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5d1f6-6e5a-4a4b-b5e1-526fafced8af service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:30.540590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:30.541630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:30.541915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:30.542273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] PATH_INFO: `/v3/users/3e589c6dfc3a49c391bae7112946e64b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:30.543983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e589c6dfc3a49c391bae7112946e64b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:30.552193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:30.810554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e589c6dfc3a49c391bae7112946e64b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:30.811182 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e589c6dfc3a49c391bae7112946e64b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:30.834490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e589c6dfc3a49c391bae7112946e64b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:30.836811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e589c6dfc3a49c391bae7112946e64b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:30.862752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e589c6dfc3a49c391bae7112946e64b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:30.977638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 894/1790] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:30 2022] DELETE /identity/v3/users/3e589c6dfc3a49c391bae7112946e64b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:30.982824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f77c9-d758-4a77-baac-d0972e999989 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:30.983571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f77c9-d758-4a77-baac-d0972e999989 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:30.984259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f77c9-d758-4a77-baac-d0972e999989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:31.013632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-131f77c9-d758-4a77-baac-d0972e999989 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:31.015299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-131f77c9-d758-4a77-baac-d0972e999989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:31.108686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:31.137807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 897/1791] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:31.254912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:31.256610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:31.407139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683c6ca-1ff2-4c3f-a1d0-73abe281d57f admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:31.422315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42a03b-7abc-48c7-84dd-7361b2812c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:31.426590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42a03b-7abc-48c7-84dd-7361b2812c45 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:31.427012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42a03b-7abc-48c7-84dd-7361b2812c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:31.427359 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42a03b-7abc-48c7-84dd-7361b2812c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:31.452816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42a03b-7abc-48c7-84dd-7361b2812c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:31.454509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42a03b-7abc-48c7-84dd-7361b2812c45 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:31.474415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 895/1792] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:31.476235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b80d6f-7d38-4790-afe7-33182285cb43 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:31.476568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b80d6f-7d38-4790-afe7-33182285cb43 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:31.476853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b80d6f-7d38-4790-afe7-33182285cb43 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:31.478601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 898/1793] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 15:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 15:56:31.591268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d50c19-a467-4a11-a44a-b6798db2c66f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:31.591560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d50c19-a467-4a11-a44a-b6798db2c66f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:31.591844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d50c19-a467-4a11-a44a-b6798db2c66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:31.627981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c3d50c19-a467-4a11-a44a-b6798db2c66f None None] MFA Rules not processed for user `312ed2b555424c6a96b3d972ffea5ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:31.629818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c3d50c19-a467-4a11-a44a-b6798db2c66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:31.763232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b80d6f-7d38-4790-afe7-33182285cb43 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:31.765731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-96b80d6f-7d38-4790-afe7-33182285cb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:31.794032 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 896/1794] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:31.800979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b80d6f-7d38-4790-afe7-33182285cb43 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:31.807609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:31.808750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:31.809177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:31.809565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] PATH_INFO: `/v3/projects/733d9d99829b40a7b4cbad1e81b1dcd8` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:31.813223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] RBAC: Authorizing `identity:delete_project(project_id=733d9d99829b40a7b4cbad1e81b1dcd8)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:31.827280 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:31.855006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d50c19-a467-4a11-a44a-b6798db2c66f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:31.859747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c3d50c19-a467-4a11-a44a-b6798db2c66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:31.886296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d50c19-a467-4a11-a44a-b6798db2c66f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:31.903071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8eb9e-aa7a-48dd-b0d7-c7ccb9bd6329 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:31.903830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8eb9e-aa7a-48dd-b0d7-c7ccb9bd6329 service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:31.904023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8eb9e-aa7a-48dd-b0d7-c7ccb9bd6329 service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:31.905721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8eb9e-aa7a-48dd-b0d7-c7ccb9bd6329 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:31.905835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:31.914822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8eb9e-aa7a-48dd-b0d7-c7ccb9bd6329 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:31.916459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8eb9e-aa7a-48dd-b0d7-c7ccb9bd6329 service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:31.931187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 897/1795] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:56:31.938525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8eb9e-aa7a-48dd-b0d7-c7ccb9bd6329 service placement] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:31.942173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-96b8eb9e-aa7a-48dd-b0d7-c7ccb9bd6329 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:31.975401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8eb9e-aa7a-48dd-b0d7-c7ccb9bd6329 service placement] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:31.992103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:31.993072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:31.993297 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:31.993570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] PATH_INFO: `/v3/projects/b9d76514996e43a8a6f90e658dd4bced` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:31.995380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d76514996e43a8a6f90e658dd4bced)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:31.997357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:32.071647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733d9d99829b40a7b4cbad1e81b1dcd8', 'request_id': 'req-2da8397c-1200-429f-9bdb-8edbdf90c77a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.083577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733d9d99829b40a7b4cbad1e81b1dcd8', 'request_id': 'req-2da8397c-1200-429f-9bdb-8edbdf90c77a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.099845 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733d9d99829b40a7b4cbad1e81b1dcd8', 'request_id': 'req-2da8397c-1200-429f-9bdb-8edbdf90c77a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.113294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:32.191047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d76514996e43a8a6f90e658dd4bced', 'request_id': 'req-489d04e7-3cb9-4d93-af57-cca93d32d435'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.207705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d76514996e43a8a6f90e658dd4bced', 'request_id': 'req-489d04e7-3cb9-4d93-af57-cca93d32d435'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.225578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d76514996e43a8a6f90e658dd4bced', 'request_id': 'req-489d04e7-3cb9-4d93-af57-cca93d32d435'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.238885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] The token cache is being invalidate because project 733d9d99829b40a7b4cbad1e81b1dcd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:32.239370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.240489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.245234 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 899/1796] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:31 2022] DELETE /identity/v3/projects/733d9d99829b40a7b4cbad1e81b1dcd8 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:32.246679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:32.252783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:32.465880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8397c-1200-429f-9bdb-8edbdf90c77a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:32.622413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:32.622413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:32.622413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:32.622413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] PATH_INFO: `/v3/users/b229b51093a748c0ad927d6cf0e821b2` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:32.625354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b229b51093a748c0ad927d6cf0e821b2)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:32.628999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:32.730327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] The token cache is being invalidate because project b9d76514996e43a8a6f90e658dd4bced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:32.733320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.737158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.741044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 898/1797] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:31 2022] DELETE /identity/v3/projects/b9d76514996e43a8a6f90e658dd4bced => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:32.745140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:32.751266 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:32.766760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d04e7-3cb9-4d93-af57-cca93d32d435 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:32.773211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:32.776918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:32.777376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:32.777871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] PATH_INFO: `/v3/users/580eaaaa79894a058ef2f0fc35466d0f` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:32.780063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] RBAC: Authorizing `identity:delete_user(user_id=580eaaaa79894a058ef2f0fc35466d0f)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:32.783479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:32.943358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b229b51093a748c0ad927d6cf0e821b2'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.944250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b229b51093a748c0ad927d6cf0e821b2'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.944930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b229b51093a748c0ad927d6cf0e821b2'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.952548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b229b51093a748c0ad927d6cf0e821b2'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:32.978331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b229b51093a748c0ad927d6cf0e821b2'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.092565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '580eaaaa79894a058ef2f0fc35466d0f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.093646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '580eaaaa79894a058ef2f0fc35466d0f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.112728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '580eaaaa79894a058ef2f0fc35466d0f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.114145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '580eaaaa79894a058ef2f0fc35466d0f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.131668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '580eaaaa79894a058ef2f0fc35466d0f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.156465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 900/1798] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:32 2022] DELETE /identity/v3/users/b229b51093a748c0ad927d6cf0e821b2 => generated 0 bytes in 903 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:33.223655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:33.225176 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:33.301306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 899/1799] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:32 2022] DELETE /identity/v3/users/580eaaaa79894a058ef2f0fc35466d0f => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:33.333810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cc332-13b0-4be8-a131-17a39926eaeb admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:33.437385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:33.439064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:33.458576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:33.465984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:33.466299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:33.466663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] PATH_INFO: `/v3/projects/faa6a40be0b6436a9453e8f981667eae` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:33.469466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa6a40be0b6436a9453e8f981667eae)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:33.471941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75537751-707f-45f6-a770-f7415f5d830c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:33.474665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:33.481671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:33.483549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:33.483848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:33.484662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] PATH_INFO: `/v3/users/391cb58098d94e07bee62fdeb306fd69` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:33.485588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] RBAC: Authorizing `identity:delete_user(user_id=391cb58098d94e07bee62fdeb306fd69)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:33.487679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:33.665749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa6a40be0b6436a9453e8f981667eae', 'request_id': 'req-938b1788-528e-47bb-a01e-d826f7fe2618'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.682608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa6a40be0b6436a9453e8f981667eae', 'request_id': 'req-938b1788-528e-47bb-a01e-d826f7fe2618'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.689912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa6a40be0b6436a9453e8f981667eae', 'request_id': 'req-938b1788-528e-47bb-a01e-d826f7fe2618'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.771270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391cb58098d94e07bee62fdeb306fd69'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.772247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391cb58098d94e07bee62fdeb306fd69'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.786959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391cb58098d94e07bee62fdeb306fd69'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.787696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391cb58098d94e07bee62fdeb306fd69'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.797500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391cb58098d94e07bee62fdeb306fd69'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.806594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] The token cache is being invalidate because project faa6a40be0b6436a9453e8f981667eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:33.806594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.806594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:33.807560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 901/1800] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:33 2022] DELETE /identity/v3/projects/faa6a40be0b6436a9453e8f981667eae => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:33.811141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:33.812712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:33.897889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 900/1801] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:33 2022] DELETE /identity/v3/users/391cb58098d94e07bee62fdeb306fd69 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:33.928467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b1788-528e-47bb-a01e-d826f7fe2618 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:33.971416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:34.024206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:34.025388 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:34.025677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:34.025962 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] PATH_INFO: `/v3/users/8839c48f2d0146f4b693ec2fb0daea3b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:34.027486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8839c48f2d0146f4b693ec2fb0daea3b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:34.030462 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:34.109887 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:34.112473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:34.131516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f4eb9-f330-442b-9d3d-cf36754ae60e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:34.138457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:34.139410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:34.139711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:34.140232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] PATH_INFO: `/v3/projects/614c2054a8ce4250a944b3acf8857e32` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:34.141763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=614c2054a8ce4250a944b3acf8857e32)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:34.143965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:34.179044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:34.297346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614c2054a8ce4250a944b3acf8857e32', 'request_id': 'req-25834a4d-276f-4b51-bbef-39dcd15b8b57'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.300562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8839c48f2d0146f4b693ec2fb0daea3b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.302250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8839c48f2d0146f4b693ec2fb0daea3b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.303257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8839c48f2d0146f4b693ec2fb0daea3b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.306345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614c2054a8ce4250a944b3acf8857e32', 'request_id': 'req-25834a4d-276f-4b51-bbef-39dcd15b8b57'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.312986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614c2054a8ce4250a944b3acf8857e32', 'request_id': 'req-25834a4d-276f-4b51-bbef-39dcd15b8b57'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.315785 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8839c48f2d0146f4b693ec2fb0daea3b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.332717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8839c48f2d0146f4b693ec2fb0daea3b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.424706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] The token cache is being invalidate because project 614c2054a8ce4250a944b3acf8857e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:34.429159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.429159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.429159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 901/1802] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:34 2022] DELETE /identity/v3/projects/614c2054a8ce4250a944b3acf8857e32 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:34.429159 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:34.429504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:34.493434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 902/1803] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:33 2022] DELETE /identity/v3/users/8839c48f2d0146f4b693ec2fb0daea3b => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:34.494347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:34.495347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25834a4d-276f-4b51-bbef-39dcd15b8b57 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:34.495677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:34.557981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:34.558872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:34.559220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:34.559575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] PATH_INFO: `/v3/projects/407115775c204537b78db32087198814` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:34.560946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=407115775c204537b78db32087198814)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:34.562913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:34.567247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef892889-3a7a-4beb-a29d-7e05835dde66 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:34.573372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:34.574341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:34.574632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:34.574936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] PATH_INFO: `/v3/projects/1d590cb287cd40bab547eefae785a7ce` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:34.576311 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d590cb287cd40bab547eefae785a7ce)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:34.579011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:34.693536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '407115775c204537b78db32087198814', 'request_id': 'req-84205ca6-de63-4d76-8719-c064295e9aee'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.706469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '407115775c204537b78db32087198814', 'request_id': 'req-84205ca6-de63-4d76-8719-c064295e9aee'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.717370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '407115775c204537b78db32087198814', 'request_id': 'req-84205ca6-de63-4d76-8719-c064295e9aee'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.723065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d590cb287cd40bab547eefae785a7ce', 'request_id': 'req-4e915217-87ac-4c63-99eb-989054ad5bf9'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.730013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d590cb287cd40bab547eefae785a7ce', 'request_id': 'req-4e915217-87ac-4c63-99eb-989054ad5bf9'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.735662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d590cb287cd40bab547eefae785a7ce', 'request_id': 'req-4e915217-87ac-4c63-99eb-989054ad5bf9'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.843578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] The token cache is being invalidate because project 407115775c204537b78db32087198814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:34.843917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.846935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-84205ca6-de63-4d76-8719-c064295e9aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.848666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 902/1804] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:34 2022] DELETE /identity/v3/projects/407115775c204537b78db32087198814 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:34.852221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf583e-385c-4303-a2b8-c9527413f631 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:34.852854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf583e-385c-4303-a2b8-c9527413f631 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:34.853319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf583e-385c-4303-a2b8-c9527413f631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:34.877429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-bacf583e-385c-4303-a2b8-c9527413f631 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:34.879480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-bacf583e-385c-4303-a2b8-c9527413f631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:34.956625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] The token cache is being invalidate because project 1d590cb287cd40bab547eefae785a7ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:34.956995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.958055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4e915217-87ac-4c63-99eb-989054ad5bf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:34.962245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 903/1806] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:34.963391 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf583e-385c-4303-a2b8-c9527413f631 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:34.965041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-bacf583e-385c-4303-a2b8-c9527413f631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:34.969492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 903/1806] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:34 2022] DELETE /identity/v3/projects/1d590cb287cd40bab547eefae785a7ce => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:34.971703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf754d-6bc1-4b4c-9c03-edc57df94af1 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:34.971996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf754d-6bc1-4b4c-9c03-edc57df94af1 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:34.972337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf754d-6bc1-4b4c-9c03-edc57df94af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:34.985011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf583e-385c-4303-a2b8-c9527413f631 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:34.992737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:34.993734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:34.994334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:35.000237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-fddf754d-6bc1-4b4c-9c03-edc57df94af1 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:35.000415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] PATH_INFO: `/v3/users/c20e581aaca94614938550ade015418c` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:35.000506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-fddf754d-6bc1-4b4c-9c03-edc57df94af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:35.000588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c20e581aaca94614938550ade015418c)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:35.000588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:35.013741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 904/1807] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:35.014839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf754d-6bc1-4b4c-9c03-edc57df94af1 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:35.016768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-fddf754d-6bc1-4b4c-9c03-edc57df94af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:35.029693 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf754d-6bc1-4b4c-9c03-edc57df94af1 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:35.035667 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:35.037076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:35.037076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:35.037265 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] PATH_INFO: `/v3/users/400cd6b39a234c5885f6262075548f48` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:35.038602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] RBAC: Authorizing `identity:delete_user(user_id=400cd6b39a234c5885f6262075548f48)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:35.040718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:35.226070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c20e581aaca94614938550ade015418c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.227103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c20e581aaca94614938550ade015418c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.240040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:35.254993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c20e581aaca94614938550ade015418c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.256350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c20e581aaca94614938550ade015418c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.264712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9bd86ba7-895a-4ce7-be6c-679d2220c604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c20e581aaca94614938550ade015418c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.265614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400cd6b39a234c5885f6262075548f48'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.267781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400cd6b39a234c5885f6262075548f48'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.269472 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400cd6b39a234c5885f6262075548f48'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.280744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400cd6b39a234c5885f6262075548f48'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.311444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-05fd26eb-2749-46bf-b3d1-49279405770e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400cd6b39a234c5885f6262075548f48'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.347804 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 904/1808] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:34 2022] DELETE /identity/v3/users/c20e581aaca94614938550ade015418c => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:35.349230 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60fc18-d22b-4adf-9106-ff42918dd9af None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:35.349522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60fc18-d22b-4adf-9106-ff42918dd9af None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:35.349802 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60fc18-d22b-4adf-9106-ff42918dd9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:35.371293 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-fc60fc18-d22b-4adf-9106-ff42918dd9af None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:35.372838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-fc60fc18-d22b-4adf-9106-ff42918dd9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:35.419278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 905/1809] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:35 2022] DELETE /identity/v3/users/400cd6b39a234c5885f6262075548f48 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:35.419278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7ae00-42e0-413c-9300-d936c08d2f0c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:35.419278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7ae00-42e0-413c-9300-d936c08d2f0c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:35.419278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7ae00-42e0-413c-9300-d936c08d2f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:35.439895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-26a7ae00-42e0-413c-9300-d936c08d2f0c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:35.456520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-26a7ae00-42e0-413c-9300-d936c08d2f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:35.456615 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 905/1810] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:35.462580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60fc18-d22b-4adf-9106-ff42918dd9af None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:35.465293 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-fc60fc18-d22b-4adf-9106-ff42918dd9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:35.481902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60fc18-d22b-4adf-9106-ff42918dd9af None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:35.524203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:35.552769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 906/1811] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:35.554302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c72af2-3362-4adf-a70c-7bdd3d57b945 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:35.554691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c72af2-3362-4adf-a70c-7bdd3d57b945 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:35.555123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c72af2-3362-4adf-a70c-7bdd3d57b945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:35.585378 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:35.586538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:35.587126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:35.587557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] PATH_INFO: `/v3/projects/e0951a2277f547669cc89a0f63ae1b0c` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:35.589428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0951a2277f547669cc89a0f63ae1b0c)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:35.590118 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-08c72af2-3362-4adf-a70c-7bdd3d57b945 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:35.592966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:35.593315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-08c72af2-3362-4adf-a70c-7bdd3d57b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:35.608589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 907/1812] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:35.609695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c72af2-3362-4adf-a70c-7bdd3d57b945 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:35.627162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c72af2-3362-4adf-a70c-7bdd3d57b945 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:35.635952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52918b7-5471-4bdc-81a1-1e53bf262f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:35.636677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52918b7-5471-4bdc-81a1-1e53bf262f8b admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:35.638632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52918b7-5471-4bdc-81a1-1e53bf262f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:35.638632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52918b7-5471-4bdc-81a1-1e53bf262f8b admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:35.638632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52918b7-5471-4bdc-81a1-1e53bf262f8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:35.638632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52918b7-5471-4bdc-81a1-1e53bf262f8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:35.640903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52918b7-5471-4bdc-81a1-1e53bf262f8b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:35.657031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 908/1813] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:35.661697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52918b7-5471-4bdc-81a1-1e53bf262f8b admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:35.673076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52918b7-5471-4bdc-81a1-1e53bf262f8b admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:35.681551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec505a-fa4e-40d9-bb7f-917adaffbffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:35.682312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec505a-fa4e-40d9-bb7f-917adaffbffa admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:35.682694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec505a-fa4e-40d9-bb7f-917adaffbffa admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:35.683152 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec505a-fa4e-40d9-bb7f-917adaffbffa admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:35.684122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec505a-fa4e-40d9-bb7f-917adaffbffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:35.685129 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec505a-fa4e-40d9-bb7f-917adaffbffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:35.686324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec505a-fa4e-40d9-bb7f-917adaffbffa admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:35.710730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 909/1814] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:35.712552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec505a-fa4e-40d9-bb7f-917adaffbffa admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:35.716674 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-acec505a-fa4e-40d9-bb7f-917adaffbffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:35.742175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0951a2277f547669cc89a0f63ae1b0c', 'request_id': 'req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.753514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0951a2277f547669cc89a0f63ae1b0c', 'request_id': 'req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.762210 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0951a2277f547669cc89a0f63ae1b0c', 'request_id': 'req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.796436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec505a-fa4e-40d9-bb7f-917adaffbffa admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:35.862033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7842c-0a23-47e6-b219-e055f784f8da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:35.862709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7842c-0a23-47e6-b219-e055f784f8da service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:35.863061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7842c-0a23-47e6-b219-e055f784f8da service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:35.863410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7842c-0a23-47e6-b219-e055f784f8da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:35.937648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] The token cache is being invalidate because project e0951a2277f547669cc89a0f63ae1b0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:35.937932 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.940745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:35.942586 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 906/1815] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:35 2022] DELETE /identity/v3/projects/e0951a2277f547669cc89a0f63ae1b0c => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:35.944010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:35.949882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:35.957275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7842c-0a23-47e6-b219-e055f784f8da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:35.959159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7842c-0a23-47e6-b219-e055f784f8da service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:35.974902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 910/1816] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:35.983007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bf13a-c09b-4c80-82bb-cb7e72873dd0 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:35.987002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7842c-0a23-47e6-b219-e055f784f8da service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:35.991898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded4483-fe8e-4f75-8f8d-6fda65345a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:35.992708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded4483-fe8e-4f75-8f8d-6fda65345a59 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:35.993425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded4483-fe8e-4f75-8f8d-6fda65345a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:35.993712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded4483-fe8e-4f75-8f8d-6fda65345a59 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:35.994519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded4483-fe8e-4f75-8f8d-6fda65345a59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:35.997425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded4483-fe8e-4f75-8f8d-6fda65345a59 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:36.007338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7842c-0a23-47e6-b219-e055f784f8da service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:36.014995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34dcd7-7cb6-4074-a2e6-38860940368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:36.016559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34dcd7-7cb6-4074-a2e6-38860940368d admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.021044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34dcd7-7cb6-4074-a2e6-38860940368d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.021044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34dcd7-7cb6-4074-a2e6-38860940368d admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.021044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34dcd7-7cb6-4074-a2e6-38860940368d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:36.021044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34dcd7-7cb6-4074-a2e6-38860940368d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:36.114972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:36.188463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 907/1817] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:35 2022] POST /identity/v3/projects => generated 407 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:36.190054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 911/1818] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:35 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:36.190891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ca8ec-41a8-4e6d-990f-c015d69869b7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.191486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ca8ec-41a8-4e6d-990f-c015d69869b7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.192926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34dcd7-7cb6-4074-a2e6-38860940368d admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:36.196599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ca8ec-41a8-4e6d-990f-c015d69869b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.229720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6b2ca8ec-41a8-4e6d-990f-c015d69869b7 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:36.235032 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6b2ca8ec-41a8-4e6d-990f-c015d69869b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:36.329613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34dcd7-7cb6-4074-a2e6-38860940368d admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:36.335275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb7ef2-3d9b-4061-8696-749be4a50a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:36.342594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb7ef2-3d9b-4061-8696-749be4a50a66 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.342594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb7ef2-3d9b-4061-8696-749be4a50a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.342594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb7ef2-3d9b-4061-8696-749be4a50a66 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.342594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb7ef2-3d9b-4061-8696-749be4a50a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:36.342594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb7ef2-3d9b-4061-8696-749be4a50a66 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:36.348928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 908/1819] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:36.348928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ca8ec-41a8-4e6d-990f-c015d69869b7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:36.351100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6b2ca8ec-41a8-4e6d-990f-c015d69869b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:36.364062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ca8ec-41a8-4e6d-990f-c015d69869b7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:36.369500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15020f-ff51-4b7a-b547-913f6ac10fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:36.370303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15020f-ff51-4b7a-b547-913f6ac10fda admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.370700 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15020f-ff51-4b7a-b547-913f6ac10fda admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.377634 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15020f-ff51-4b7a-b547-913f6ac10fda admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.377634 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15020f-ff51-4b7a-b547-913f6ac10fda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:36.377634 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15020f-ff51-4b7a-b547-913f6ac10fda admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:36.398799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:36.467478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 912/1820] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:36 2022] POST /identity/v3/users => generated 304 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:36.483463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb7ef2-3d9b-4061-8696-749be4a50a66 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:36.490261 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-aebb7ef2-3d9b-4061-8696-749be4a50a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:36.501160 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 909/1821] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:36 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:36.506636 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d9fc8-54e2-462d-be0a-d13721b6903d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.506970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d9fc8-54e2-462d-be0a-d13721b6903d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.508432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d9fc8-54e2-462d-be0a-d13721b6903d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.523386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb7ef2-3d9b-4061-8696-749be4a50a66 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:36.532996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:36.532996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.532996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.532996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] PATH_INFO: `/v3/projects/358f5c0b2a374a1eaa7b36041f48c765` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.533835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] RBAC: Authorizing `identity:delete_project(project_id=358f5c0b2a374a1eaa7b36041f48c765)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:36.534822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:36.553568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1e7d9fc8-54e2-462d-be0a-d13721b6903d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:36.555289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1e7d9fc8-54e2-462d-be0a-d13721b6903d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:36.592105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 910/1822] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:36.593944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d9fc8-54e2-462d-be0a-d13721b6903d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:36.607290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d9fc8-54e2-462d-be0a-d13721b6903d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:36.628044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9681018-91c2-4c75-bbb0-c64649e66b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:36.628044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9681018-91c2-4c75-bbb0-c64649e66b81 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.628044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9681018-91c2-4c75-bbb0-c64649e66b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.628044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9681018-91c2-4c75-bbb0-c64649e66b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.628901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9681018-91c2-4c75-bbb0-c64649e66b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:36.628901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9681018-91c2-4c75-bbb0-c64649e66b81 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:36.656558 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 911/1823] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:36.664545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9681018-91c2-4c75-bbb0-c64649e66b81 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:36.678564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9681018-91c2-4c75-bbb0-c64649e66b81 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:36.692745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '358f5c0b2a374a1eaa7b36041f48c765', 'request_id': 'req-114ebae5-316a-4dc3-8119-ed225558347c'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:36.702012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '358f5c0b2a374a1eaa7b36041f48c765', 'request_id': 'req-114ebae5-316a-4dc3-8119-ed225558347c'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:36.717951 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '358f5c0b2a374a1eaa7b36041f48c765', 'request_id': 'req-114ebae5-316a-4dc3-8119-ed225558347c'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:36.801880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07675813-0e6d-4b4c-8a88-e8c9c191968e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:36.804857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07675813-0e6d-4b4c-8a88-e8c9c191968e admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.804857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07675813-0e6d-4b4c-8a88-e8c9c191968e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.804857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07675813-0e6d-4b4c-8a88-e8c9c191968e admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.804857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07675813-0e6d-4b4c-8a88-e8c9c191968e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:36.806027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07675813-0e6d-4b4c-8a88-e8c9c191968e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:36.820609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 912/1824] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:36.821906 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07675813-0e6d-4b4c-8a88-e8c9c191968e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:36.835097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] The token cache is being invalidate because project 358f5c0b2a374a1eaa7b36041f48c765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:36.835395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:36.838656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:36.845176 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 913/1825] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:36 2022] DELETE /identity/v3/projects/358f5c0b2a374a1eaa7b36041f48c765 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:36.846380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07675813-0e6d-4b4c-8a88-e8c9c191968e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:36.848778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:36.851487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:36.859127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c6987-9f07-4363-b129-8d254f78ebc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:36.859833 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c6987-9f07-4363-b129-8d254f78ebc4 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.860123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c6987-9f07-4363-b129-8d254f78ebc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.860819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c6987-9f07-4363-b129-8d254f78ebc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.861218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c6987-9f07-4363-b129-8d254f78ebc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:36.862566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c6987-9f07-4363-b129-8d254f78ebc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:36.863762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c6987-9f07-4363-b129-8d254f78ebc4 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:36.884943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ebae5-316a-4dc3-8119-ed225558347c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:36.891461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 913/1826] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:36.893100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c6987-9f07-4363-b129-8d254f78ebc4 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:36.902574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-db1c6987-9f07-4363-b129-8d254f78ebc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:36.904455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef87101-9abc-412b-a8d0-756908000109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:36.905314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef87101-9abc-412b-a8d0-756908000109 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.905707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef87101-9abc-412b-a8d0-756908000109 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.905986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef87101-9abc-412b-a8d0-756908000109 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.906731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef87101-9abc-412b-a8d0-756908000109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:36.909185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef87101-9abc-412b-a8d0-756908000109 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:36.934605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c6987-9f07-4363-b129-8d254f78ebc4 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:36.938984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 914/1827] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:36.940702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf835e2-0633-49db-993f-099dc80d9494 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.941557 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf835e2-0633-49db-993f-099dc80d9494 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.941841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf835e2-0633-49db-993f-099dc80d9494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.950655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782d98a-6e77-4456-8daa-245d2d258089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:36.951395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782d98a-6e77-4456-8daa-245d2d258089 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:36.951679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782d98a-6e77-4456-8daa-245d2d258089 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:36.951953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782d98a-6e77-4456-8daa-245d2d258089 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:36.952715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782d98a-6e77-4456-8daa-245d2d258089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:36.954336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782d98a-6e77-4456-8daa-245d2d258089 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:36.978357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 914/1828] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:36.985467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2bf835e2-0633-49db-993f-099dc80d9494 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:36.986717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782d98a-6e77-4456-8daa-245d2d258089 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:36.988053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2bf835e2-0633-49db-993f-099dc80d9494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:36.991046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3782d98a-6e77-4456-8daa-245d2d258089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:37.015249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 915/1829] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:37.018340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf835e2-0633-49db-993f-099dc80d9494 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:37.030210 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782d98a-6e77-4456-8daa-245d2d258089 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:37.040147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59ae7c-c5ef-4ca6-9c05-d9444fe98765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:37.046149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59ae7c-c5ef-4ca6-9c05-d9444fe98765 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:37.046149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59ae7c-c5ef-4ca6-9c05-d9444fe98765 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:37.046149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59ae7c-c5ef-4ca6-9c05-d9444fe98765 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:37.046149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59ae7c-c5ef-4ca6-9c05-d9444fe98765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:37.046149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59ae7c-c5ef-4ca6-9c05-d9444fe98765 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:37.046743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf835e2-0633-49db-993f-099dc80d9494 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:37.054596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4a471-7677-4bdf-b754-9f9005b0409c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:37.056874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a471-7677-4bdf-b754-9f9005b0409c admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:37.057274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a471-7677-4bdf-b754-9f9005b0409c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:37.057941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4a471-7677-4bdf-b754-9f9005b0409c admin admin] PATH_INFO: `/v3/projects/1cf850896a0b4d00a895599b1d515b96/users/973dd100274c46be8f13365b3ba8c971/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:37.081577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4a471-7677-4bdf-b754-9f9005b0409c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf850896a0b4d00a895599b1d515b96, user_id=973dd100274c46be8f13365b3ba8c971, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:37.084510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4a471-7677-4bdf-b754-9f9005b0409c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:37.180128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 916/1830] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:37 2022] PUT /identity/v3/projects/1cf850896a0b4d00a895599b1d515b96/users/973dd100274c46be8f13365b3ba8c971/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:37.180935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4a471-7677-4bdf-b754-9f9005b0409c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:37.194008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 915/1831] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:36 2022] POST /identity/v3/projects => generated 385 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:37.194008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae752a2a-5965-4e6c-a180-8bab21f46a70 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:37.194008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae752a2a-5965-4e6c-a180-8bab21f46a70 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:37.196353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae752a2a-5965-4e6c-a180-8bab21f46a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:37.248127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ae752a2a-5965-4e6c-a180-8bab21f46a70 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:37.255999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ae752a2a-5965-4e6c-a180-8bab21f46a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:37.400269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4a471-7677-4bdf-b754-9f9005b0409c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:37.411512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 916/1832] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:37.413880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a94b9-d75d-4660-984b-efae4a855585 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:37.414391 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a94b9-d75d-4660-984b-efae4a855585 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:37.414867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a94b9-d75d-4660-984b-efae4a855585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:37.483265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-676a94b9-d75d-4660-984b-efae4a855585 None None] MFA Rules not processed for user `973dd100274c46be8f13365b3ba8c971`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:37.485096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-676a94b9-d75d-4660-984b-efae4a855585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:37.496098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b01c89-2783-4425-aff7-6d3d6a0907a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:37.497312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b01c89-2783-4425-aff7-6d3d6a0907a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:37.497604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b01c89-2783-4425-aff7-6d3d6a0907a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:37.497936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b01c89-2783-4425-aff7-6d3d6a0907a3 admin admin] PATH_INFO: `/v3/projects/5ad816536d0d4f0db4470de71153b49e/users/c7edefc4c50b4b4e86eae49c0b208253/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:37.534488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b01c89-2783-4425-aff7-6d3d6a0907a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad816536d0d4f0db4470de71153b49e, user_id=c7edefc4c50b4b4e86eae49c0b208253, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:37.537394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b01c89-2783-4425-aff7-6d3d6a0907a3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:37.567006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:37.609457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 917/1833] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:37 2022] PUT /identity/v3/projects/5ad816536d0d4f0db4470de71153b49e/users/c7edefc4c50b4b4e86eae49c0b208253/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:37.610027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b01c89-2783-4425-aff7-6d3d6a0907a3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:37.612931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 917/1834] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:37.614300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a94b9-d75d-4660-984b-efae4a855585 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:37.616956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-676a94b9-d75d-4660-984b-efae4a855585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:37.638651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a94b9-d75d-4660-984b-efae4a855585 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:37.747689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:37.748881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:37.750727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:37.751029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] PATH_INFO: `/v3/projects/0907ecc679014a37ac423ec2721c3178` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:37.754631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0907ecc679014a37ac423ec2721c3178)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:37.760688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:37.807251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b01c89-2783-4425-aff7-6d3d6a0907a3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:37.817817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b1065-fae0-49c7-b053-b0a4a2d75022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:37.817817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b1065-fae0-49c7-b053-b0a4a2d75022 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:37.818129 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b1065-fae0-49c7-b053-b0a4a2d75022 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:37.818338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b1065-fae0-49c7-b053-b0a4a2d75022 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:37.819536 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b1065-fae0-49c7-b053-b0a4a2d75022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:37.821704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b1065-fae0-49c7-b053-b0a4a2d75022 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:38.580094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0907ecc679014a37ac423ec2721c3178', 'request_id': 'req-b4565f86-180e-4be4-8433-1ddcfacdfb86'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:38.586991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0907ecc679014a37ac423ec2721c3178', 'request_id': 'req-b4565f86-180e-4be4-8433-1ddcfacdfb86'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:38.592968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:38.594437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0907ecc679014a37ac423ec2721c3178', 'request_id': 'req-b4565f86-180e-4be4-8433-1ddcfacdfb86'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:38.599987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 918/1835] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:37 2022] POST /identity/v3/users => generated 293 bytes in 990 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:38.601402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605e052-656f-4a3e-8380-9bd0591ec815 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:38.601695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605e052-656f-4a3e-8380-9bd0591ec815 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:38.602002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605e052-656f-4a3e-8380-9bd0591ec815 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:38.603401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 919/1836] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 15:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:56:38.604141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605e052-656f-4a3e-8380-9bd0591ec815 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:38.669624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605e052-656f-4a3e-8380-9bd0591ec815 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:38.674861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a7ef9-cb90-491a-acf5-eb9deed8ae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:38.675598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7ef9-cb90-491a-acf5-eb9deed8ae14 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:38.675880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7ef9-cb90-491a-acf5-eb9deed8ae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:38.676155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7ef9-cb90-491a-acf5-eb9deed8ae14 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:38.676921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a7ef9-cb90-491a-acf5-eb9deed8ae14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:38.677234 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a7ef9-cb90-491a-acf5-eb9deed8ae14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:38.678847 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a7ef9-cb90-491a-acf5-eb9deed8ae14 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:38.692623 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 920/1837] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:38.693834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff98bb-44c9-4f8b-80a9-0f3201a84f89 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:38.694122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff98bb-44c9-4f8b-80a9-0f3201a84f89 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:38.694399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff98bb-44c9-4f8b-80a9-0f3201a84f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:38.715989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] The token cache is being invalidate because project 0907ecc679014a37ac423ec2721c3178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:38.716275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:38.716970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:38.718513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 918/1838] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:37 2022] DELETE /identity/v3/projects/0907ecc679014a37ac423ec2721c3178 => generated 0 bytes in 1105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:38.719256 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:38.720713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:38.724254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-dbff98bb-44c9-4f8b-80a9-0f3201a84f89 None None] MFA Rules not processed for user `c7edefc4c50b4b4e86eae49c0b208253`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:38.725482 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-dbff98bb-44c9-4f8b-80a9-0f3201a84f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:38.732882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4565f86-180e-4be4-8433-1ddcfacdfb86 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:38.738045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9d5dd-e7a2-44d6-8bc3-2d130977cb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:38.738708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9d5dd-e7a2-44d6-8bc3-2d130977cb43 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:38.738981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9d5dd-e7a2-44d6-8bc3-2d130977cb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:38.739261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9d5dd-e7a2-44d6-8bc3-2d130977cb43 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:38.739955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9d5dd-e7a2-44d6-8bc3-2d130977cb43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:38.741887 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9d5dd-e7a2-44d6-8bc3-2d130977cb43 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:38.790203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 921/1839] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:38.791041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff98bb-44c9-4f8b-80a9-0f3201a84f89 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:38.792435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-dbff98bb-44c9-4f8b-80a9-0f3201a84f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:38.805863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff98bb-44c9-4f8b-80a9-0f3201a84f89 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:38.810532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab20ff5-d44f-4a90-866c-7c0ef236eac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:38.812023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab20ff5-d44f-4a90-866c-7c0ef236eac5 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:38.812023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab20ff5-d44f-4a90-866c-7c0ef236eac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:38.812023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab20ff5-d44f-4a90-866c-7c0ef236eac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:38.812522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab20ff5-d44f-4a90-866c-7c0ef236eac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:38.814123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab20ff5-d44f-4a90-866c-7c0ef236eac5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:38.823802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 922/1840] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:38.825007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f49779-8c5b-4ee9-a4ff-128d2761d6db None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:38.825338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f49779-8c5b-4ee9-a4ff-128d2761d6db None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:38.825619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f49779-8c5b-4ee9-a4ff-128d2761d6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:38.838535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 919/1841] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:38 2022] POST /identity/v3/projects => generated 407 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:38.840210 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9d5dd-e7a2-44d6-8bc3-2d130977cb43 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:38.842947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a8c9d5dd-e7a2-44d6-8bc3-2d130977cb43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:38.852509 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b5f49779-8c5b-4ee9-a4ff-128d2761d6db None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:38.853740 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b5f49779-8c5b-4ee9-a4ff-128d2761d6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:38.915871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9d5dd-e7a2-44d6-8bc3-2d130977cb43 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:38.920633 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4df61-d666-4745-b8e0-e57afef62af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:38.921303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4df61-d666-4745-b8e0-e57afef62af7 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:38.934716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4df61-d666-4745-b8e0-e57afef62af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:38.934716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4df61-d666-4745-b8e0-e57afef62af7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:38.934716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4df61-d666-4745-b8e0-e57afef62af7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:38.934716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4df61-d666-4745-b8e0-e57afef62af7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:38.935348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 923/1842] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 15:56:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:38.935348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139a380-5faa-43b3-a78c-bfc277343b9d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:38.935348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139a380-5faa-43b3-a78c-bfc277343b9d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:38.935348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139a380-5faa-43b3-a78c-bfc277343b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:38.956156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0139a380-5faa-43b3-a78c-bfc277343b9d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:38.956156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0139a380-5faa-43b3-a78c-bfc277343b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:38.962045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 924/1843] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:38.962641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139a380-5faa-43b3-a78c-bfc277343b9d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:38.964575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0139a380-5faa-43b3-a78c-bfc277343b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:38.980483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139a380-5faa-43b3-a78c-bfc277343b9d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:38.981981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369cd8-afaf-437f-be43-be1e3cc5de87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:38.982647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369cd8-afaf-437f-be43-be1e3cc5de87 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:38.983223 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369cd8-afaf-437f-be43-be1e3cc5de87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:38.983510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369cd8-afaf-437f-be43-be1e3cc5de87 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:38.984207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60369cd8-afaf-437f-be43-be1e3cc5de87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:38.987025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60369cd8-afaf-437f-be43-be1e3cc5de87 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.007486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 920/1844] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:38 2022] POST /identity/v3/projects => generated 365 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:39.008880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4df61-d666-4745-b8e0-e57afef62af7 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.074148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4df61-d666-4745-b8e0-e57afef62af7 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.078865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d402ef-c273-490c-ab62-de89e136de2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.079537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d402ef-c273-490c-ab62-de89e136de2b admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.079821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d402ef-c273-490c-ab62-de89e136de2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.080151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d402ef-c273-490c-ab62-de89e136de2b admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.080915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d402ef-c273-490c-ab62-de89e136de2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.082502 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d402ef-c273-490c-ab62-de89e136de2b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.089250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 925/1845] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:38 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:39.090342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369cd8-afaf-437f-be43-be1e3cc5de87 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.096756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 921/1846] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:39.098203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d402ef-c273-490c-ab62-de89e136de2b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.101116 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-89d402ef-c273-490c-ab62-de89e136de2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:39.167523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369cd8-afaf-437f-be43-be1e3cc5de87 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.172553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d03a3c-22ee-4aad-9617-25a626abebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.173884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d03a3c-22ee-4aad-9617-25a626abebb4 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.174276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d03a3c-22ee-4aad-9617-25a626abebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.175041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d03a3c-22ee-4aad-9617-25a626abebb4 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.175864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d03a3c-22ee-4aad-9617-25a626abebb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.178331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d03a3c-22ee-4aad-9617-25a626abebb4 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.186922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d402ef-c273-490c-ab62-de89e136de2b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.255066 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ea37e-3156-4dfe-a70d-1c84e90d150c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.255699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ea37e-3156-4dfe-a70d-1c84e90d150c service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.256029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ea37e-3156-4dfe-a70d-1c84e90d150c service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.256311 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ea37e-3156-4dfe-a70d-1c84e90d150c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.266320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ea37e-3156-4dfe-a70d-1c84e90d150c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.269004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ea37e-3156-4dfe-a70d-1c84e90d150c service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.283938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 922/1847] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:39.295288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad158735-867c-4d79-830d-2ddc8e086a61 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.295288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad158735-867c-4d79-830d-2ddc8e086a61 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.295288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad158735-867c-4d79-830d-2ddc8e086a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.299183 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 926/1848] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:39 2022] POST /identity/v3/users => generated 310 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:39.300626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d03a3c-22ee-4aad-9617-25a626abebb4 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.318670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ad158735-867c-4d79-830d-2ddc8e086a61 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:39.320010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ad158735-867c-4d79-830d-2ddc8e086a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:39.337979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 923/1849] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:39.338141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d03a3c-22ee-4aad-9617-25a626abebb4 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.343044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad158735-867c-4d79-830d-2ddc8e086a61 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.347616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50e7eb-3fe6-423d-b46f-787efb43c7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.348308 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50e7eb-3fe6-423d-b46f-787efb43c7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.348653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50e7eb-3fe6-423d-b46f-787efb43c7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.348981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50e7eb-3fe6-423d-b46f-787efb43c7f0 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.351174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50e7eb-3fe6-423d-b46f-787efb43c7f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.355016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad158735-867c-4d79-830d-2ddc8e086a61 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.357436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50e7eb-3fe6-423d-b46f-787efb43c7f0 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.360318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ec1d8-280c-4100-8b86-ec3f8d425376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.361241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ec1d8-280c-4100-8b86-ec3f8d425376 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.361631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ec1d8-280c-4100-8b86-ec3f8d425376 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.362147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ec1d8-280c-4100-8b86-ec3f8d425376 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.363016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ec1d8-280c-4100-8b86-ec3f8d425376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.365086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ec1d8-280c-4100-8b86-ec3f8d425376 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.517114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 927/1850] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:39 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:39.518849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 924/1851] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:39 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:39.520166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50e7eb-3fe6-423d-b46f-787efb43c7f0 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.520291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd393b-d8d1-4c17-a7f0-2a05f1b5f0fa None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.520607 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd393b-d8d1-4c17-a7f0-2a05f1b5f0fa None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.520889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd393b-d8d1-4c17-a7f0-2a05f1b5f0fa None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.522645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 925/1852] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 15:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:56:39.523414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd393b-d8d1-4c17-a7f0-2a05f1b5f0fa None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.536153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50e7eb-3fe6-423d-b46f-787efb43c7f0 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.541450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2581d34-09e9-47af-8908-d917ffd6e68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.542467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2581d34-09e9-47af-8908-d917ffd6e68f admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.542750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2581d34-09e9-47af-8908-d917ffd6e68f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.543027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2581d34-09e9-47af-8908-d917ffd6e68f admin admin] PATH_INFO: `/v3/projects/66bb3acd2f6842db84d86d246f0414f4/users/a0f41b54baa44d528cc6723be16dc01c/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.543959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd393b-d8d1-4c17-a7f0-2a05f1b5f0fa None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.550610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820acb1-86b4-4e6b-a6e4-f69349d71855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.551299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820acb1-86b4-4e6b-a6e4-f69349d71855 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.551584 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820acb1-86b4-4e6b-a6e4-f69349d71855 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.551858 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820acb1-86b4-4e6b-a6e4-f69349d71855 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.552689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820acb1-86b4-4e6b-a6e4-f69349d71855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.554756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820acb1-86b4-4e6b-a6e4-f69349d71855 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.567899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2581d34-09e9-47af-8908-d917ffd6e68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bb3acd2f6842db84d86d246f0414f4, user_id=a0f41b54baa44d528cc6723be16dc01c, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.571881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2581d34-09e9-47af-8908-d917ffd6e68f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.580193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:39.595386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 926/1853] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:39.595386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820acb1-86b4-4e6b-a6e4-f69349d71855 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.607648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820acb1-86b4-4e6b-a6e4-f69349d71855 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.613400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6154c53-0b8a-4b19-b680-89c805f1485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.614145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6154c53-0b8a-4b19-b680-89c805f1485b admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.614419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6154c53-0b8a-4b19-b680-89c805f1485b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.615227 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6154c53-0b8a-4b19-b680-89c805f1485b admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.615980 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6154c53-0b8a-4b19-b680-89c805f1485b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:39.616428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6154c53-0b8a-4b19-b680-89c805f1485b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.621756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6154c53-0b8a-4b19-b680-89c805f1485b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.634502 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 927/1854] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:39.635140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6154c53-0b8a-4b19-b680-89c805f1485b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.650015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6154c53-0b8a-4b19-b680-89c805f1485b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.656410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427a77-d3de-423a-8913-86b9fe3954e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.657414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427a77-d3de-423a-8913-86b9fe3954e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.657703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427a77-d3de-423a-8913-86b9fe3954e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.657981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427a77-d3de-423a-8913-86b9fe3954e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.658772 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427a77-d3de-423a-8913-86b9fe3954e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.661259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427a77-d3de-423a-8913-86b9fe3954e0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.678851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 928/1856] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:39.678851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427a77-d3de-423a-8913-86b9fe3954e0 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.679446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 928/1856] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:39 2022] PUT /identity/v3/projects/66bb3acd2f6842db84d86d246f0414f4/users/a0f41b54baa44d528cc6723be16dc01c/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:39.681775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dffc52-7f29-4691-948d-b69a8cf6842b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.682284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dffc52-7f29-4691-948d-b69a8cf6842b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.682762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dffc52-7f29-4691-948d-b69a8cf6842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.691521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427a77-d3de-423a-8913-86b9fe3954e0 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.717492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-70dffc52-7f29-4691-948d-b69a8cf6842b None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:39.718843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-70dffc52-7f29-4691-948d-b69a8cf6842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:39.787430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36807317-f6cf-4867-9865-4b9c4b90472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.788112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36807317-f6cf-4867-9865-4b9c4b90472d admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.788417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36807317-f6cf-4867-9865-4b9c4b90472d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.788697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36807317-f6cf-4867-9865-4b9c4b90472d admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.789587 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36807317-f6cf-4867-9865-4b9c4b90472d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.791226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36807317-f6cf-4867-9865-4b9c4b90472d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.799113 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 929/1857] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 15:56:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:39.799965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dffc52-7f29-4691-948d-b69a8cf6842b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.806326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 929/1858] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:39.806912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36807317-f6cf-4867-9865-4b9c4b90472d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.811791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dffc52-7f29-4691-948d-b69a8cf6842b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.818833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240fb2e-5a75-465c-858c-4547860e35c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.819154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240fb2e-5a75-465c-858c-4547860e35c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.819453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240fb2e-5a75-465c-858c-4547860e35c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.819786 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240fb2e-5a75-465c-858c-4547860e35c7 admin admin] PATH_INFO: `/v3/projects/f22d7c545b5a4241b309496095e6134f/users/32c9bd0f8e6b4d9caffc7b766dc406be/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.824330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36807317-f6cf-4867-9865-4b9c4b90472d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:39.829630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c946de-032b-4643-b51d-b00b29c5b7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:39.831136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c946de-032b-4643-b51d-b00b29c5b7ea admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.834715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c946de-032b-4643-b51d-b00b29c5b7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.834715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c946de-032b-4643-b51d-b00b29c5b7ea admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.834715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c946de-032b-4643-b51d-b00b29c5b7ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.834715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c946de-032b-4643-b51d-b00b29c5b7ea admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.839066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240fb2e-5a75-465c-858c-4547860e35c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22d7c545b5a4241b309496095e6134f, user_id=32c9bd0f8e6b4d9caffc7b766dc406be, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:39.846487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240fb2e-5a75-465c-858c-4547860e35c7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:39.905247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 930/1859] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:39 2022] PUT /identity/v3/projects/f22d7c545b5a4241b309496095e6134f/users/32c9bd0f8e6b4d9caffc7b766dc406be/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:39.906270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240fb2e-5a75-465c-858c-4547860e35c7 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:39.930205 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 930/1860] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:39 2022] POST /identity/v3/projects => generated 355 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:39.931543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccb2a3-4cee-41dc-8702-649af06bfc57 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:39.931948 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccb2a3-4cee-41dc-8702-649af06bfc57 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:39.932242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ccb2a3-4cee-41dc-8702-649af06bfc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:39.959548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-53ccb2a3-4cee-41dc-8702-649af06bfc57 None None] MFA Rules not processed for user `a0f41b54baa44d528cc6723be16dc01c`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:39.960912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-53ccb2a3-4cee-41dc-8702-649af06bfc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:39.974071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240fb2e-5a75-465c-858c-4547860e35c7 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.028660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 931/1861] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:40.030163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccb2a3-4cee-41dc-8702-649af06bfc57 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.033690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce49721-2626-4f74-98b4-0589af7559a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.034275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce49721-2626-4f74-98b4-0589af7559a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.034582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce49721-2626-4f74-98b4-0589af7559a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.034865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce49721-2626-4f74-98b4-0589af7559a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.035604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce49721-2626-4f74-98b4-0589af7559a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.037272 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce49721-2626-4f74-98b4-0589af7559a8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.041643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ccb2a3-4cee-41dc-8702-649af06bfc57 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.046510 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d80f7-29fa-4305-b18c-ee822ed7ca35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.047532 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d80f7-29fa-4305-b18c-ee822ed7ca35 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.047860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d80f7-29fa-4305-b18c-ee822ed7ca35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.048246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d80f7-29fa-4305-b18c-ee822ed7ca35 admin admin] PATH_INFO: `/v3/projects/4d75c7e9426a414d80dc36fdf78ca892/users/2956f505794a4daea9cfa296b1a5d0e3/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.048377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 931/1862] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:40.049864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce49721-2626-4f74-98b4-0589af7559a8 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.065305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d80f7-29fa-4305-b18c-ee822ed7ca35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d75c7e9426a414d80dc36fdf78ca892, user_id=2956f505794a4daea9cfa296b1a5d0e3, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.067947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d80f7-29fa-4305-b18c-ee822ed7ca35 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.122312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce49721-2626-4f74-98b4-0589af7559a8 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.128781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9f567-541e-4e8c-96b1-dd00f1baa954 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.129436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9f567-541e-4e8c-96b1-dd00f1baa954 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.129716 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9f567-541e-4e8c-96b1-dd00f1baa954 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.129989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9f567-541e-4e8c-96b1-dd00f1baa954 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.139111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9f567-541e-4e8c-96b1-dd00f1baa954 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.140750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9f567-541e-4e8c-96b1-dd00f1baa954 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.151485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 932/1863] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:40.152270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9f567-541e-4e8c-96b1-dd00f1baa954 service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.163513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9f567-541e-4e8c-96b1-dd00f1baa954 service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.168285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f346b-182f-4fd3-9d63-957171da4c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.169053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f346b-182f-4fd3-9d63-957171da4c50 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.169343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f346b-182f-4fd3-9d63-957171da4c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.171363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f346b-182f-4fd3-9d63-957171da4c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.171363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f346b-182f-4fd3-9d63-957171da4c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.172075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f346b-182f-4fd3-9d63-957171da4c50 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.183426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 933/1864] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:40.184400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f346b-182f-4fd3-9d63-957171da4c50 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.195272 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f346b-182f-4fd3-9d63-957171da4c50 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.197635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 932/1865] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:40 2022] PUT /identity/v3/projects/4d75c7e9426a414d80dc36fdf78ca892/users/2956f505794a4daea9cfa296b1a5d0e3/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:40.200205 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d80f7-29fa-4305-b18c-ee822ed7ca35 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.254688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58806977-d0eb-4ce9-98b3-f0fb38cc07a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.255374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58806977-d0eb-4ce9-98b3-f0fb38cc07a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.255661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58806977-d0eb-4ce9-98b3-f0fb38cc07a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.255945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58806977-d0eb-4ce9-98b3-f0fb38cc07a9 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.256767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58806977-d0eb-4ce9-98b3-f0fb38cc07a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.258730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58806977-d0eb-4ce9-98b3-f0fb38cc07a9 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.266269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d80f7-29fa-4305-b18c-ee822ed7ca35 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.271097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36415f-6d7d-4cf2-953a-935621e16adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.271757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36415f-6d7d-4cf2-953a-935621e16adc admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.272303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36415f-6d7d-4cf2-953a-935621e16adc admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.272609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36415f-6d7d-4cf2-953a-935621e16adc admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.273557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36415f-6d7d-4cf2-953a-935621e16adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.275461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36415f-6d7d-4cf2-953a-935621e16adc admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.376613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 934/1866] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:40 2022] POST /identity/v3/users => generated 278 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:40.376613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58806977-d0eb-4ce9-98b3-f0fb38cc07a9 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.384080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58806977-d0eb-4ce9-98b3-f0fb38cc07a9 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.388770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f6ee-a572-4083-8115-a706ab060347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.389634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f6ee-a572-4083-8115-a706ab060347 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.389924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f6ee-a572-4083-8115-a706ab060347 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.390211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f6ee-a572-4083-8115-a706ab060347 admin admin] PATH_INFO: `/v3/projects/49b30d8ba86242e7bce27d36af163443/users/32031e7c80f04de1a6cce41a33d961e2/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.405680 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22f6ee-a572-4083-8115-a706ab060347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b30d8ba86242e7bce27d36af163443, user_id=32031e7c80f04de1a6cce41a33d961e2, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.406044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 933/1867] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:40 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:40.406938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36415f-6d7d-4cf2-953a-935621e16adc admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.409373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22f6ee-a572-4083-8115-a706ab060347 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.489743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36415f-6d7d-4cf2-953a-935621e16adc admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.494659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e518cd-6afd-4ab3-b2fd-45d5760f394a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.495557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e518cd-6afd-4ab3-b2fd-45d5760f394a admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.495838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e518cd-6afd-4ab3-b2fd-45d5760f394a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.496180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e518cd-6afd-4ab3-b2fd-45d5760f394a admin admin] PATH_INFO: `/v3/projects/f22d7c545b5a4241b309496095e6134f/users/32c9bd0f8e6b4d9caffc7b766dc406be/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.498325 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e518cd-6afd-4ab3-b2fd-45d5760f394a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22d7c545b5a4241b309496095e6134f, user_id=32c9bd0f8e6b4d9caffc7b766dc406be, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.501019 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e518cd-6afd-4ab3-b2fd-45d5760f394a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.519860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 934/1869] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:40 2022] PUT /identity/v3/projects/f22d7c545b5a4241b309496095e6134f/users/32c9bd0f8e6b4d9caffc7b766dc406be/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:40.519860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e518cd-6afd-4ab3-b2fd-45d5760f394a admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.520401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 935/1869] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:40 2022] PUT /identity/v3/projects/49b30d8ba86242e7bce27d36af163443/users/32031e7c80f04de1a6cce41a33d961e2/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:40.521547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f6ee-a572-4083-8115-a706ab060347 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.596491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e518cd-6afd-4ab3-b2fd-45d5760f394a admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.601444 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1a9c0-678c-45b6-931b-3f52fbc59617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.602112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1a9c0-678c-45b6-931b-3f52fbc59617 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.602395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1a9c0-678c-45b6-931b-3f52fbc59617 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.602669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1a9c0-678c-45b6-931b-3f52fbc59617 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.603443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1a9c0-678c-45b6-931b-3f52fbc59617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.605152 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1a9c0-678c-45b6-931b-3f52fbc59617 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.614194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f6ee-a572-4083-8115-a706ab060347 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.617851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 935/1870] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:40.618724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1a9c0-678c-45b6-931b-3f52fbc59617 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.620063 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57e90c-dd7b-4e03-86b1-ed38e86a4aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.620989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57e90c-dd7b-4e03-86b1-ed38e86a4aac admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.621357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57e90c-dd7b-4e03-86b1-ed38e86a4aac admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.621708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57e90c-dd7b-4e03-86b1-ed38e86a4aac admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.622809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57e90c-dd7b-4e03-86b1-ed38e86a4aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.624278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57e90c-dd7b-4e03-86b1-ed38e86a4aac admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.630803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1a9c0-678c-45b6-931b-3f52fbc59617 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.635716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24a1bd-37f4-4171-a139-4b618fd92cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.636507 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24a1bd-37f4-4171-a139-4b618fd92cae admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.636866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24a1bd-37f4-4171-a139-4b618fd92cae admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.637215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24a1bd-37f4-4171-a139-4b618fd92cae admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.638403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24a1bd-37f4-4171-a139-4b618fd92cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.640354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24a1bd-37f4-4171-a139-4b618fd92cae admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.644123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 936/1871] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:40.645635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b9f16-d8b7-4d05-a676-234ae77017fd None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.646001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b9f16-d8b7-4d05-a676-234ae77017fd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.646287 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b9f16-d8b7-4d05-a676-234ae77017fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.674643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-af4b9f16-d8b7-4d05-a676-234ae77017fd None None] MFA Rules not processed for user `32031e7c80f04de1a6cce41a33d961e2`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:40.675848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-af4b9f16-d8b7-4d05-a676-234ae77017fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:40.748182 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 936/1872] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:40 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:40.750198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48559fc0-f145-41a1-b58e-a61e607cd6e1 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.750198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48559fc0-f145-41a1-b58e-a61e607cd6e1 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.750372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48559fc0-f145-41a1-b58e-a61e607cd6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.754497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 937/1873] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:40.755824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b9f16-d8b7-4d05-a676-234ae77017fd None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.772268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b9f16-d8b7-4d05-a676-234ae77017fd None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.781103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eb808-0dd3-4fba-84e2-b2bfb97c174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.782012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eb808-0dd3-4fba-84e2-b2bfb97c174c admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.782296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eb808-0dd3-4fba-84e2-b2bfb97c174c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.782575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eb808-0dd3-4fba-84e2-b2bfb97c174c admin admin] PATH_INFO: `/v3/projects/4d75c7e9426a414d80dc36fdf78ca892/users/2956f505794a4daea9cfa296b1a5d0e3/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.785384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50eb808-0dd3-4fba-84e2-b2bfb97c174c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d75c7e9426a414d80dc36fdf78ca892, user_id=2956f505794a4daea9cfa296b1a5d0e3, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.789303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50eb808-0dd3-4fba-84e2-b2bfb97c174c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.789431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-48559fc0-f145-41a1-b58e-a61e607cd6e1 None None] MFA Rules not processed for user `32c9bd0f8e6b4d9caffc7b766dc406be`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:40.790784 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-48559fc0-f145-41a1-b58e-a61e607cd6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:40.802930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 938/1874] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:40 2022] PUT /identity/v3/projects/4d75c7e9426a414d80dc36fdf78ca892/users/2956f505794a4daea9cfa296b1a5d0e3/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:40.803678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eb808-0dd3-4fba-84e2-b2bfb97c174c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.890478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 937/1875] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:40 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:40.890478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48559fc0-f145-41a1-b58e-a61e607cd6e1 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.900852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48559fc0-f145-41a1-b58e-a61e607cd6e1 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.909385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a9c30-0614-4559-bdc0-17aa46ad8d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.910495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a9c30-0614-4559-bdc0-17aa46ad8d9e admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.916982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a9c30-0614-4559-bdc0-17aa46ad8d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.916982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a9c30-0614-4559-bdc0-17aa46ad8d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.917172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eb808-0dd3-4fba-84e2-b2bfb97c174c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.917257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a9c30-0614-4559-bdc0-17aa46ad8d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.917257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a9c30-0614-4559-bdc0-17aa46ad8d9e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.919656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e0939-937b-4660-9245-0b42255b2408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.920371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e0939-937b-4660-9245-0b42255b2408 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.920886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e0939-937b-4660-9245-0b42255b2408 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.921060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e0939-937b-4660-9245-0b42255b2408 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.921848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e0939-937b-4660-9245-0b42255b2408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.924268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e0939-937b-4660-9245-0b42255b2408 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.944258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 939/1876] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:40.944258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e0939-937b-4660-9245-0b42255b2408 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:40.948436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 938/1877] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:40.950188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2e26f-168b-4042-b636-b1771478d674 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.951211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2e26f-168b-4042-b636-b1771478d674 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.951211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2e26f-168b-4042-b636-b1771478d674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.961172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e0939-937b-4660-9245-0b42255b2408 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:40.969210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be3c74-b21d-45de-8a4c-4be84114a3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:40.970084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be3c74-b21d-45de-8a4c-4be84114a3fc admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:40.971764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be3c74-b21d-45de-8a4c-4be84114a3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:40.972059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be3c74-b21d-45de-8a4c-4be84114a3fc admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:40.972806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be3c74-b21d-45de-8a4c-4be84114a3fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:40.975170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be3c74-b21d-45de-8a4c-4be84114a3fc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:40.981985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:40.989257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-09c2e26f-168b-4042-b636-b1771478d674 None None] MFA Rules not processed for user `2956f505794a4daea9cfa296b1a5d0e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:40.990880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-09c2e26f-168b-4042-b636-b1771478d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:41.093899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 940/1878] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:40 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:41.093899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be3c74-b21d-45de-8a4c-4be84114a3fc admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.094156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 939/1879] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:41.094156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2e26f-168b-4042-b636-b1771478d674 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.096598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-44be3c74-b21d-45de-8a4c-4be84114a3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:41.112834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2e26f-168b-4042-b636-b1771478d674 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:41.181156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa016c-e7d3-461f-a8de-286b2bca040e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:41.182059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa016c-e7d3-461f-a8de-286b2bca040e admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.182420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa016c-e7d3-461f-a8de-286b2bca040e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.182794 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa016c-e7d3-461f-a8de-286b2bca040e admin admin] PATH_INFO: `/v3/projects/c1d9de4cc70d4be8a3201bd4efd09351/users/c2c32310b7064390a628d8e190ea6027/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.194928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be3c74-b21d-45de-8a4c-4be84114a3fc admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:41.199865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2d894-9145-45cb-92bc-d4b4cac12058 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:41.201476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2d894-9145-45cb-92bc-d4b4cac12058 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.201946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2d894-9145-45cb-92bc-d4b4cac12058 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.202058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa016c-e7d3-461f-a8de-286b2bca040e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d9de4cc70d4be8a3201bd4efd09351, user_id=c2c32310b7064390a628d8e190ea6027, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:41.202383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2d894-9145-45cb-92bc-d4b4cac12058 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.204769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa016c-e7d3-461f-a8de-286b2bca040e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:41.277825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2d894-9145-45cb-92bc-d4b4cac12058 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:41.283787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2d894-9145-45cb-92bc-d4b4cac12058 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:41.296124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 941/1880] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:41 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:41.297611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2d894-9145-45cb-92bc-d4b4cac12058 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.311071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 940/1881] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:41 2022] PUT /identity/v3/projects/c1d9de4cc70d4be8a3201bd4efd09351/users/c2c32310b7064390a628d8e190ea6027/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:41.311071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa016c-e7d3-461f-a8de-286b2bca040e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.311275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2d894-9145-45cb-92bc-d4b4cac12058 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:41.403638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55316e8a-b45c-4181-ad8a-a63791e88d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:41.404700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55316e8a-b45c-4181-ad8a-a63791e88d82 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.404989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55316e8a-b45c-4181-ad8a-a63791e88d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.405268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55316e8a-b45c-4181-ad8a-a63791e88d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.406543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55316e8a-b45c-4181-ad8a-a63791e88d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:41.408490 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55316e8a-b45c-4181-ad8a-a63791e88d82 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:41.412228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa016c-e7d3-461f-a8de-286b2bca040e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:41.424848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7720e6-727c-440f-adab-5f5d26962bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:41.429813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7720e6-727c-440f-adab-5f5d26962bed admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.429813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7720e6-727c-440f-adab-5f5d26962bed admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.429813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7720e6-727c-440f-adab-5f5d26962bed admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.430102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 942/1882] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:41.430191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7720e6-727c-440f-adab-5f5d26962bed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:41.430280 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55316e8a-b45c-4181-ad8a-a63791e88d82 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.430445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-55316e8a-b45c-4181-ad8a-a63791e88d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:41.431325 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7720e6-727c-440f-adab-5f5d26962bed admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:41.494818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55316e8a-b45c-4181-ad8a-a63791e88d82 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:41.498587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a986f-0177-46bb-b438-e13cfcf472ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:41.499245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a986f-0177-46bb-b438-e13cfcf472ac service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.499537 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a986f-0177-46bb-b438-e13cfcf472ac service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.499821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a986f-0177-46bb-b438-e13cfcf472ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.540236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 941/1883] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:41 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:41.541822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bb225-2ce1-457c-ad89-1781452a0cd2 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.542116 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bb225-2ce1-457c-ad89-1781452a0cd2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.542435 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bb225-2ce1-457c-ad89-1781452a0cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.566729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a986f-0177-46bb-b438-e13cfcf472ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:41.568107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a986f-0177-46bb-b438-e13cfcf472ac service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:41.571804 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-5d1bb225-2ce1-457c-ad89-1781452a0cd2 None None] MFA Rules not processed for user `c2c32310b7064390a628d8e190ea6027`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:41.573080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5d1bb225-2ce1-457c-ad89-1781452a0cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:41.578075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 943/1884] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:41.579052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a986f-0177-46bb-b438-e13cfcf472ac service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.590648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a986f-0177-46bb-b438-e13cfcf472ac service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:41.594361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbe448-29d0-4b0f-9b4b-5aa1a74eefa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:41.595343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbe448-29d0-4b0f-9b4b-5aa1a74eefa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.595705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbe448-29d0-4b0f-9b4b-5aa1a74eefa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.595995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbe448-29d0-4b0f-9b4b-5aa1a74eefa0 admin admin] PATH_INFO: `/v3/projects/e5a13bda976b4bd0868f3923c80d3894/users/7ebf54444a574b5f8fe311c3176d2373/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.617323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbe448-29d0-4b0f-9b4b-5aa1a74eefa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5a13bda976b4bd0868f3923c80d3894, user_id=7ebf54444a574b5f8fe311c3176d2373, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:41.620030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbe448-29d0-4b0f-9b4b-5aa1a74eefa0 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:41.664790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 942/1885] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:41 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:41.666691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bb225-2ce1-457c-ad89-1781452a0cd2 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.679186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bb225-2ce1-457c-ad89-1781452a0cd2 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:41.684627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a056592-a4a5-4045-b46d-2513e9e9e5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:41.684789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056592-a4a5-4045-b46d-2513e9e9e5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.685163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056592-a4a5-4045-b46d-2513e9e9e5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.685341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056592-a4a5-4045-b46d-2513e9e9e5d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.686073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a056592-a4a5-4045-b46d-2513e9e9e5d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:41.688035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a056592-a4a5-4045-b46d-2513e9e9e5d2 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:41.700963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 943/1886] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:41.702119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a056592-a4a5-4045-b46d-2513e9e9e5d2 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.714174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a056592-a4a5-4045-b46d-2513e9e9e5d2 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:41.720817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 944/1887] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:41 2022] PUT /identity/v3/projects/e5a13bda976b4bd0868f3923c80d3894/users/7ebf54444a574b5f8fe311c3176d2373/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:41.721644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbe448-29d0-4b0f-9b4b-5aa1a74eefa0 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.793162 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49395b9c-b930-47de-a78f-8887991395a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:41.793851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49395b9c-b930-47de-a78f-8887991395a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.794204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49395b9c-b930-47de-a78f-8887991395a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.794657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49395b9c-b930-47de-a78f-8887991395a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.795511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49395b9c-b930-47de-a78f-8887991395a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:41.797492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49395b9c-b930-47de-a78f-8887991395a0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:41.804944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbe448-29d0-4b0f-9b4b-5aa1a74eefa0 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:41.810276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e96f80-1823-4512-8cb1-8fc54adce7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:41.811497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e96f80-1823-4512-8cb1-8fc54adce7e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.812166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e96f80-1823-4512-8cb1-8fc54adce7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.812166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e96f80-1823-4512-8cb1-8fc54adce7e2 admin admin] PATH_INFO: `/v3/projects/3b885a22765d41fea6392f3caf1cf65c/users/737ec2b8936740f9880eb05ca6c25ac1/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.828958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e96f80-1823-4512-8cb1-8fc54adce7e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b885a22765d41fea6392f3caf1cf65c, user_id=737ec2b8936740f9880eb05ca6c25ac1, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:41.832107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e96f80-1823-4512-8cb1-8fc54adce7e2 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:41.875927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 944/1888] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:41 2022] POST /identity/v3/projects => generated 357 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:41.878016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0840cc-9070-4557-ab47-520565e620c1 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.878310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0840cc-9070-4557-ab47-520565e620c1 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.878589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0840cc-9070-4557-ab47-520565e620c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.904999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6b0840cc-9070-4557-ab47-520565e620c1 None None] MFA Rules not processed for user `7ebf54444a574b5f8fe311c3176d2373`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:41.906208 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 945/1889] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:41 2022] PUT /identity/v3/projects/3b885a22765d41fea6392f3caf1cf65c/users/737ec2b8936740f9880eb05ca6c25ac1/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:41.906336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6b0840cc-9070-4557-ab47-520565e620c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:41.912826 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e96f80-1823-4512-8cb1-8fc54adce7e2 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.964117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e96f80-1823-4512-8cb1-8fc54adce7e2 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:41.969931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597be9ad-9e76-42d7-aae2-e89e9e9af695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:41.970589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597be9ad-9e76-42d7-aae2-e89e9e9af695 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:41.981581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597be9ad-9e76-42d7-aae2-e89e9e9af695 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:41.981581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597be9ad-9e76-42d7-aae2-e89e9e9af695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:41.981909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 945/1890] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:41.981909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0840cc-9070-4557-ab47-520565e620c1 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:41.982040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-597be9ad-9e76-42d7-aae2-e89e9e9af695 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:42.039267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597be9ad-9e76-42d7-aae2-e89e9e9af695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.040086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0840cc-9070-4557-ab47-520565e620c1 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.040783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597be9ad-9e76-42d7-aae2-e89e9e9af695 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.044860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339fd35-ad57-4d3c-b0a1-0aa3d19b50e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.045700 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339fd35-ad57-4d3c-b0a1-0aa3d19b50e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.045984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339fd35-ad57-4d3c-b0a1-0aa3d19b50e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.046264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339fd35-ad57-4d3c-b0a1-0aa3d19b50e8 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.046954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339fd35-ad57-4d3c-b0a1-0aa3d19b50e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.048890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339fd35-ad57-4d3c-b0a1-0aa3d19b50e8 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.050132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 946/1891] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:41 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:42.051087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597be9ad-9e76-42d7-aae2-e89e9e9af695 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.061426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597be9ad-9e76-42d7-aae2-e89e9e9af695 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.066798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575f5a5-eaf4-43e5-b4f1-92dbecc7ba44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.067463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575f5a5-eaf4-43e5-b4f1-92dbecc7ba44 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.067772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575f5a5-eaf4-43e5-b4f1-92dbecc7ba44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.068033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575f5a5-eaf4-43e5-b4f1-92dbecc7ba44 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.068917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575f5a5-eaf4-43e5-b4f1-92dbecc7ba44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.070649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575f5a5-eaf4-43e5-b4f1-92dbecc7ba44 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.085213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 947/1892] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:42.085213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575f5a5-eaf4-43e5-b4f1-92dbecc7ba44 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.094866 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575f5a5-eaf4-43e5-b4f1-92dbecc7ba44 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.101301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b89630-9879-47d6-83fe-c57395ce4f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.102527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b89630-9879-47d6-83fe-c57395ce4f43 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.102527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b89630-9879-47d6-83fe-c57395ce4f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.102673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b89630-9879-47d6-83fe-c57395ce4f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.104207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b89630-9879-47d6-83fe-c57395ce4f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.106267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b89630-9879-47d6-83fe-c57395ce4f43 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.185379 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 946/1893] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:41 2022] POST /identity/v3/users => generated 285 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:42.192503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339fd35-ad57-4d3c-b0a1-0aa3d19b50e8 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.203448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339fd35-ad57-4d3c-b0a1-0aa3d19b50e8 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.214604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc65cb-b60f-43c2-870e-5ef533346175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.216671 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 948/1894] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:42 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:42.217764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b89630-9879-47d6-83fe-c57395ce4f43 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.218762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc65cb-b60f-43c2-870e-5ef533346175 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.220710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc65cb-b60f-43c2-870e-5ef533346175 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.221879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc65cb-b60f-43c2-870e-5ef533346175 admin admin] PATH_INFO: `/v3/projects/3b885a22765d41fea6392f3caf1cf65c/users/737ec2b8936740f9880eb05ca6c25ac1/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.224610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc65cb-b60f-43c2-870e-5ef533346175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b885a22765d41fea6392f3caf1cf65c, user_id=737ec2b8936740f9880eb05ca6c25ac1, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.227659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc65cb-b60f-43c2-870e-5ef533346175 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.240285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 947/1895] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:42 2022] PUT /identity/v3/projects/3b885a22765d41fea6392f3caf1cf65c/users/737ec2b8936740f9880eb05ca6c25ac1/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:42.240870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc65cb-b60f-43c2-870e-5ef533346175 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.320938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b89630-9879-47d6-83fe-c57395ce4f43 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.323931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc65cb-b60f-43c2-870e-5ef533346175 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.326033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1567b-73be-430d-91bc-6c765cba2abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.331349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1567b-73be-430d-91bc-6c765cba2abb admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.331349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1567b-73be-430d-91bc-6c765cba2abb admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.331349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1567b-73be-430d-91bc-6c765cba2abb admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.331349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1567b-73be-430d-91bc-6c765cba2abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.331655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa8b07-a360-44e9-ae62-19d314b04044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.331769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1567b-73be-430d-91bc-6c765cba2abb admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.331841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa8b07-a360-44e9-ae62-19d314b04044 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.331841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa8b07-a360-44e9-ae62-19d314b04044 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.331841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa8b07-a360-44e9-ae62-19d314b04044 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.332317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa8b07-a360-44e9-ae62-19d314b04044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.334206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa8b07-a360-44e9-ae62-19d314b04044 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.342156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 949/1896] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:42.347381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8e78-e543-44bc-ac20-84fb913245c8 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.347745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8e78-e543-44bc-ac20-84fb913245c8 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.348095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e8e78-e543-44bc-ac20-84fb913245c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.352254 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=72,keystone:DELETE=26,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:42.377011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-550e8e78-e543-44bc-ac20-84fb913245c8 None None] MFA Rules not processed for user `737ec2b8936740f9880eb05ca6c25ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:42.378604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-550e8e78-e543-44bc-ac20-84fb913245c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:42.443482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 948/1897] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:42 2022] POST /identity/v3/users => generated 300 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:42.444698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa8b07-a360-44e9-ae62-19d314b04044 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.456315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa8b07-a360-44e9-ae62-19d314b04044 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.462747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb250e-59b8-483d-9403-911fbbbbd618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.464561 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 950/1898] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:42.464697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb250e-59b8-483d-9403-911fbbbbd618 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.464697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb250e-59b8-483d-9403-911fbbbbd618 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.464697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb250e-59b8-483d-9403-911fbbbbd618 admin admin] PATH_INFO: `/v3/projects/92bb0361afd84fbabea16ecb91b1d57c/users/2be1b77c3942405a9201e80f4268778b/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.470322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e8e78-e543-44bc-ac20-84fb913245c8 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.476279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e8e78-e543-44bc-ac20-84fb913245c8 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.482356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381da482-08a1-47ed-9bfd-c3285b14e731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.483143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381da482-08a1-47ed-9bfd-c3285b14e731 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.483628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381da482-08a1-47ed-9bfd-c3285b14e731 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.483735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb250e-59b8-483d-9403-911fbbbbd618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92bb0361afd84fbabea16ecb91b1d57c, user_id=2be1b77c3942405a9201e80f4268778b, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.484030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381da482-08a1-47ed-9bfd-c3285b14e731 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.484886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381da482-08a1-47ed-9bfd-c3285b14e731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.487038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb250e-59b8-483d-9403-911fbbbbd618 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.492627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381da482-08a1-47ed-9bfd-c3285b14e731 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.510242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 951/1899] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:42.513154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381da482-08a1-47ed-9bfd-c3285b14e731 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.530052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381da482-08a1-47ed-9bfd-c3285b14e731 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.541684 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14802301-c35d-46e5-bb5e-c7e5e3aaf181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.542614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14802301-c35d-46e5-bb5e-c7e5e3aaf181 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.542887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14802301-c35d-46e5-bb5e-c7e5e3aaf181 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.543282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14802301-c35d-46e5-bb5e-c7e5e3aaf181 admin admin] PATH_INFO: `/v3/projects/6391ff96f67241529957a14766c72a91/users/2d9ca85f7ba64069823ee9667ffe9f02/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.560378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14802301-c35d-46e5-bb5e-c7e5e3aaf181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6391ff96f67241529957a14766c72a91, user_id=2d9ca85f7ba64069823ee9667ffe9f02, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.563114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14802301-c35d-46e5-bb5e-c7e5e3aaf181 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.582291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 949/1900] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:42 2022] PUT /identity/v3/projects/92bb0361afd84fbabea16ecb91b1d57c/users/2be1b77c3942405a9201e80f4268778b/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:42.583362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb250e-59b8-483d-9403-911fbbbbd618 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.624881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 952/1901] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:42 2022] PUT /identity/v3/projects/6391ff96f67241529957a14766c72a91/users/2d9ca85f7ba64069823ee9667ffe9f02/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:42.625855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14802301-c35d-46e5-bb5e-c7e5e3aaf181 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.660157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb250e-59b8-483d-9403-911fbbbbd618 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.700713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14802301-c35d-46e5-bb5e-c7e5e3aaf181 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.729769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f6227-4555-47a8-8c8f-6ac1fe24fb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.729769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f6227-4555-47a8-8c8f-6ac1fe24fb7e admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.730066 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d41e84-5022-4fc0-91b3-f523edc31c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.730519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f6227-4555-47a8-8c8f-6ac1fe24fb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.730519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f6227-4555-47a8-8c8f-6ac1fe24fb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.730666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d41e84-5022-4fc0-91b3-f523edc31c36 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.730666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d41e84-5022-4fc0-91b3-f523edc31c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.730788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f6227-4555-47a8-8c8f-6ac1fe24fb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.730854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d41e84-5022-4fc0-91b3-f523edc31c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.730854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d41e84-5022-4fc0-91b3-f523edc31c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.730980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f6227-4555-47a8-8c8f-6ac1fe24fb7e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.731047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d41e84-5022-4fc0-91b3-f523edc31c36 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.740563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 950/1902] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:56:42.742783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 953/1903] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:42.744597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d41e84-5022-4fc0-91b3-f523edc31c36 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.744729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f6227-4555-47a8-8c8f-6ac1fe24fb7e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.755300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d41e84-5022-4fc0-91b3-f523edc31c36 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.756418 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f6227-4555-47a8-8c8f-6ac1fe24fb7e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.759886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d516f19-f441-4cce-8327-f26c77e197ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.760961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516f19-f441-4cce-8327-f26c77e197ea admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.761242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516f19-f441-4cce-8327-f26c77e197ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.761337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b4b4a-9b4b-486b-98a7-16f7df0b4ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.761518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d516f19-f441-4cce-8327-f26c77e197ea admin admin] PATH_INFO: `/v3/projects/92bb0361afd84fbabea16ecb91b1d57c/users/2be1b77c3942405a9201e80f4268778b/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.762211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b4b4a-9b4b-486b-98a7-16f7df0b4ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.762501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b4b4a-9b4b-486b-98a7-16f7df0b4ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.763051 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b4b4a-9b4b-486b-98a7-16f7df0b4ea7 admin admin] PATH_INFO: `/v3/projects/6391ff96f67241529957a14766c72a91/users/2d9ca85f7ba64069823ee9667ffe9f02/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.763600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d516f19-f441-4cce-8327-f26c77e197ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=92bb0361afd84fbabea16ecb91b1d57c, user_id=2be1b77c3942405a9201e80f4268778b, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.765120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b4b4a-9b4b-486b-98a7-16f7df0b4ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6391ff96f67241529957a14766c72a91, user_id=2d9ca85f7ba64069823ee9667ffe9f02, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:42.766267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d516f19-f441-4cce-8327-f26c77e197ea admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.767844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b4b4a-9b4b-486b-98a7-16f7df0b4ea7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:42.777498 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 951/1904] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:42 2022] PUT /identity/v3/projects/92bb0361afd84fbabea16ecb91b1d57c/users/2be1b77c3942405a9201e80f4268778b/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:42.778292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d516f19-f441-4cce-8327-f26c77e197ea admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:42.779354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 954/1905] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:42 2022] PUT /identity/v3/projects/6391ff96f67241529957a14766c72a91/users/2d9ca85f7ba64069823ee9667ffe9f02/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:42.788835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df8221-6f60-434c-8edd-59927151a5e7 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.789003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df8221-6f60-434c-8edd-59927151a5e7 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.789291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df8221-6f60-434c-8edd-59927151a5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.822650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-58df8221-6f60-434c-8edd-59927151a5e7 None None] MFA Rules not processed for user `2be1b77c3942405a9201e80f4268778b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:42.823875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-58df8221-6f60-434c-8edd-59927151a5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:42.858404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d516f19-f441-4cce-8327-f26c77e197ea admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:42.936301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcef98e-6612-461e-a380-640a5fb8e765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:42.937011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef98e-6612-461e-a380-640a5fb8e765 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.937296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef98e-6612-461e-a380-640a5fb8e765 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.937575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef98e-6612-461e-a380-640a5fb8e765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:42.971605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 955/1906] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:42 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:42.973033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a102ec6-dfa2-4836-8f07-fce4208c5f55 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:42.973901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a102ec6-dfa2-4836-8f07-fce4208c5f55 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:42.973901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a102ec6-dfa2-4836-8f07-fce4208c5f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:43.001825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-1a102ec6-dfa2-4836-8f07-fce4208c5f55 None None] MFA Rules not processed for user `2d9ca85f7ba64069823ee9667ffe9f02`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:43.003619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1a102ec6-dfa2-4836-8f07-fce4208c5f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:43.020955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcef98e-6612-461e-a380-640a5fb8e765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:43.022974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcef98e-6612-461e-a380-640a5fb8e765 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:43.034348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 952/1907] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:43.035216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcef98e-6612-461e-a380-640a5fb8e765 service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:43.094696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 956/1908] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:43.095558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a102ec6-dfa2-4836-8f07-fce4208c5f55 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:43.105332 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a102ec6-dfa2-4836-8f07-fce4208c5f55 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:43.109114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568c29c-65ed-45f8-a1a6-d6325cbeed01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:43.109792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568c29c-65ed-45f8-a1a6-d6325cbeed01 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:43.110079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568c29c-65ed-45f8-a1a6-d6325cbeed01 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:43.110356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568c29c-65ed-45f8-a1a6-d6325cbeed01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:43.113097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcef98e-6612-461e-a380-640a5fb8e765 service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:43.120947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b06df-e549-45a6-bb3b-b3b22f8b4b3c tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2956f505794a4daea9cfa296b1a5d0e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d75c7e9426a414d80dc36fdf78ca892', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:43.123656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568c29c-65ed-45f8-a1a6-d6325cbeed01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:43.123780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b06df-e549-45a6-bb3b-b3b22f8b4b3c tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:43.123780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b06df-e549-45a6-bb3b-b3b22f8b4b3c tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:43.123780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b06df-e549-45a6-bb3b-b3b22f8b4b3c tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:43.124028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568c29c-65ed-45f8-a1a6-d6325cbeed01 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:43.124119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b06df-e549-45a6-bb3b-b3b22f8b4b3c tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:43.124119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b06df-e549-45a6-bb3b-b3b22f8b4b3c tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:43.125816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b06df-e549-45a6-bb3b-b3b22f8b4b3c tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:43.135005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 957/1909] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:43 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:43.147596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568c29c-65ed-45f8-a1a6-d6325cbeed01 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:43.154119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 953/1910] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:43.160867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568c29c-65ed-45f8-a1a6-d6325cbeed01 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:43.165980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f4a29-df19-4ac1-82b5-9a6f57a185be tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2956f505794a4daea9cfa296b1a5d0e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d75c7e9426a414d80dc36fdf78ca892', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:43.166178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b06df-e549-45a6-bb3b-b3b22f8b4b3c tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:43.166708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f4a29-df19-4ac1-82b5-9a6f57a185be tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:43.166938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f4a29-df19-4ac1-82b5-9a6f57a185be tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:43.167219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f4a29-df19-4ac1-82b5-9a6f57a185be tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:43.167922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f4a29-df19-4ac1-82b5-9a6f57a185be tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:43.170023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f4a29-df19-4ac1-82b5-9a6f57a185be tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:43.178067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b06df-e549-45a6-bb3b-b3b22f8b4b3c tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:43.182133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4082457-0c18-4517-8081-f1f2e496ad7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:43.182798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4082457-0c18-4517-8081-f1f2e496ad7a service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:43.183093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4082457-0c18-4517-8081-f1f2e496ad7a service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:43.183410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4082457-0c18-4517-8081-f1f2e496ad7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:43.193920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4082457-0c18-4517-8081-f1f2e496ad7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:43.195435 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4082457-0c18-4517-8081-f1f2e496ad7a service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:43.205559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 954/1911] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:56:43.267678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 958/1912] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:43 2022] POST /identity/v3/projects => generated 354 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:43.273935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4082457-0c18-4517-8081-f1f2e496ad7a service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:43.284840 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4082457-0c18-4517-8081-f1f2e496ad7a service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:43.350758 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979eb6e-4dff-4c7c-94cc-e1f7c0f119d3 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2956f505794a4daea9cfa296b1a5d0e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d75c7e9426a414d80dc36fdf78ca892', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:43.354601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979eb6e-4dff-4c7c-94cc-e1f7c0f119d3 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:43.354601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979eb6e-4dff-4c7c-94cc-e1f7c0f119d3 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:43.354601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979eb6e-4dff-4c7c-94cc-e1f7c0f119d3 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:43.354601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979eb6e-4dff-4c7c-94cc-e1f7c0f119d3 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:43.354601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979eb6e-4dff-4c7c-94cc-e1f7c0f119d3 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:43.355007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979eb6e-4dff-4c7c-94cc-e1f7c0f119d3 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:43.377400 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 955/1913] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:43.397685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f4a29-df19-4ac1-82b5-9a6f57a185be tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:43.435954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979eb6e-4dff-4c7c-94cc-e1f7c0f119d3 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:43.449824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979eb6e-4dff-4c7c-94cc-e1f7c0f119d3 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:43.457731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2956f505794a4daea9cfa296b1a5d0e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d75c7e9426a414d80dc36fdf78ca892', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:43.459062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:43.459308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:43.459973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] PATH_INFO: `/v3/projects/71de236e410e4c39a63eb81fcc81534b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:43.461504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: Authorizing `identity:delete_project(project_id=71de236e410e4c39a63eb81fcc81534b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:43.464069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:43.477284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f4a29-df19-4ac1-82b5-9a6f57a185be tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:43.482758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb830fa3-2bc8-425e-b2c6-c779f82df8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:43.484616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb830fa3-2bc8-425e-b2c6-c779f82df8eb service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:43.484910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb830fa3-2bc8-425e-b2c6-c779f82df8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:43.485189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb830fa3-2bc8-425e-b2c6-c779f82df8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:43.566679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb830fa3-2bc8-425e-b2c6-c779f82df8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:43.568508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb830fa3-2bc8-425e-b2c6-c779f82df8eb service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:43.578955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 959/1914] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:43 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:43.579799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb830fa3-2bc8-425e-b2c6-c779f82df8eb service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:43.589497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb830fa3-2bc8-425e-b2c6-c779f82df8eb service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:43.606695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e50694-85e4-4c32-9766-2d9c125e908b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:43.607524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e50694-85e4-4c32-9766-2d9c125e908b service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:43.607813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e50694-85e4-4c32-9766-2d9c125e908b service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:43.608086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e50694-85e4-4c32-9766-2d9c125e908b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:43.625751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71de236e410e4c39a63eb81fcc81534b', 'request_id': 'req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:43.639224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71de236e410e4c39a63eb81fcc81534b', 'request_id': 'req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:43.656411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71de236e410e4c39a63eb81fcc81534b', 'request_id': 'req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:43.703633 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e50694-85e4-4c32-9766-2d9c125e908b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:43.705952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e50694-85e4-4c32-9766-2d9c125e908b service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:43.716834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:43.726015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 960/1915] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:43.743299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] The token cache is being invalidate because project 71de236e410e4c39a63eb81fcc81534b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:43.744902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:43.744902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bc3ce0ac-4e0d-4a6c-927c-8f6208b6df21 tempest-VolumeQuotaClassesTest-876662018 tempest-VolumeQuotaClassesTest-876662018-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:43.746997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 956/1916] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:43 2022] DELETE /identity/v3/projects/71de236e410e4c39a63eb81fcc81534b => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:44.804705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e50694-85e4-4c32-9766-2d9c125e908b service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:44.901303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e50694-85e4-4c32-9766-2d9c125e908b service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:44.951436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:44.987779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:44.987779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:44.987779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:44.987779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] PATH_INFO: `/v3/users/c7edefc4c50b4b4e86eae49c0b208253` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:44.988973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7edefc4c50b4b4e86eae49c0b208253)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:44.991443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:45.279512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7edefc4c50b4b4e86eae49c0b208253'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:45.280522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7edefc4c50b4b4e86eae49c0b208253'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:45.281450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7edefc4c50b4b4e86eae49c0b208253'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:45.300703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7edefc4c50b4b4e86eae49c0b208253'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:45.339528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7edefc4c50b4b4e86eae49c0b208253'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:45.445578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 961/1917] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:44 2022] DELETE /identity/v3/users/c7edefc4c50b4b4e86eae49c0b208253 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:45.450399 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c965779-6e33-4770-9f6e-7892545dfcb4 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:45.450725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c965779-6e33-4770-9f6e-7892545dfcb4 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:45.451061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c965779-6e33-4770-9f6e-7892545dfcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:45.471009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-4c965779-6e33-4770-9f6e-7892545dfcb4 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:45.473377 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-4c965779-6e33-4770-9f6e-7892545dfcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:45.542539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 957/1918] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:45.552895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:45.554630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:45.613307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66032ff8-1ab4-4a3b-b1f2-9f2c6d092f37 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:45.616865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d9c03-1ff6-45cb-ada8-b8009bc1b331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:45.617519 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9c03-1ff6-45cb-ada8-b8009bc1b331 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:45.617807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9c03-1ff6-45cb-ada8-b8009bc1b331 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:45.618087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9c03-1ff6-45cb-ada8-b8009bc1b331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:45.627436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d9c03-1ff6-45cb-ada8-b8009bc1b331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:45.628911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d9c03-1ff6-45cb-ada8-b8009bc1b331 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:45.638254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 962/1919] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:46.106592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c965779-6e33-4770-9f6e-7892545dfcb4 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:46.108050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-4c965779-6e33-4770-9f6e-7892545dfcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:46.120122 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c965779-6e33-4770-9f6e-7892545dfcb4 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:46.125726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:46.126540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:46.126846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:46.127100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] PATH_INFO: `/v3/projects/5ad816536d0d4f0db4470de71153b49e` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:46.128345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad816536d0d4f0db4470de71153b49e)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:46.130304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:46.298339 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad816536d0d4f0db4470de71153b49e', 'request_id': 'req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:46.305229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad816536d0d4f0db4470de71153b49e', 'request_id': 'req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:46.310766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad816536d0d4f0db4470de71153b49e', 'request_id': 'req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:46.336066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d9c03-1ff6-45cb-ada8-b8009bc1b331 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:46.337584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-aa7d9c03-1ff6-45cb-ada8-b8009bc1b331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:46.410794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:46.417870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d9c03-1ff6-45cb-ada8-b8009bc1b331 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:46.423199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] The token cache is being invalidate because project 5ad816536d0d4f0db4470de71153b49e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:46.423751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:46.424691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:46.426375 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 958/1920] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:46 2022] DELETE /identity/v3/projects/5ad816536d0d4f0db4470de71153b49e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:46.430313 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:46.432023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:46.479242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc7349-0f91-4b4c-8c55-9884956dd142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:46.480027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7349-0f91-4b4c-8c55-9884956dd142 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:46.480425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7349-0f91-4b4c-8c55-9884956dd142 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:46.480777 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7349-0f91-4b4c-8c55-9884956dd142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:46.482858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-87fc7349-0f91-4b4c-8c55-9884956dd142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:46.511145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbb9df-ba9c-4c7b-b15c-9f8237735db5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:46.516044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:46.516964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:46.517246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:46.517672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] PATH_INFO: `/v3/users/2956f505794a4daea9cfa296b1a5d0e3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:46.519912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2956f505794a4daea9cfa296b1a5d0e3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:46.522353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:46.548498 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc7349-0f91-4b4c-8c55-9884956dd142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:46.550199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc7349-0f91-4b4c-8c55-9884956dd142 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:46.561101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 963/1921] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:56:46.736275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2956f505794a4daea9cfa296b1a5d0e3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:46.737765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2956f505794a4daea9cfa296b1a5d0e3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:46.751632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2956f505794a4daea9cfa296b1a5d0e3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:46.753124 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2956f505794a4daea9cfa296b1a5d0e3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:46.762337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-422e0d6a-6ee5-4526-834c-46f2cc6b0130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2956f505794a4daea9cfa296b1a5d0e3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:46.898357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 959/1922] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:46 2022] DELETE /identity/v3/users/2956f505794a4daea9cfa296b1a5d0e3 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:47.364567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc7349-0f91-4b4c-8c55-9884956dd142 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:47.379291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc7349-0f91-4b4c-8c55-9884956dd142 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:47.479994 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:47.481031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:47.481484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:47.481931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] PATH_INFO: `/v3/projects/4d75c7e9426a414d80dc36fdf78ca892` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:47.483405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d75c7e9426a414d80dc36fdf78ca892)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:47.485574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:47.608607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d75c7e9426a414d80dc36fdf78ca892', 'request_id': 'req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:47.615498 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d75c7e9426a414d80dc36fdf78ca892', 'request_id': 'req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:47.620981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d75c7e9426a414d80dc36fdf78ca892', 'request_id': 'req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:47.681368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] The token cache is being invalidate because project 4d75c7e9426a414d80dc36fdf78ca892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:47.681657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:47.682267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0cdb7485-7cb7-4c25-bc17-7c28b0462bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:47.683844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 964/1923] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:47 2022] DELETE /identity/v3/projects/4d75c7e9426a414d80dc36fdf78ca892 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:47.692414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b39a4f-fd82-4411-880d-bee5aed1e179 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:47.692883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b39a4f-fd82-4411-880d-bee5aed1e179 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:47.692993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b39a4f-fd82-4411-880d-bee5aed1e179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:47.711124 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a4b39a4f-fd82-4411-880d-bee5aed1e179 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:47.712309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a4b39a4f-fd82-4411-880d-bee5aed1e179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:47.769229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 960/1924] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:47.778242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227cf8f-3791-4b05-a9da-95ff4f275702 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:47.778533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227cf8f-3791-4b05-a9da-95ff4f275702 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:47.778833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227cf8f-3791-4b05-a9da-95ff4f275702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:47.783311 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:47.798677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-1227cf8f-3791-4b05-a9da-95ff4f275702 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:47.799891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1227cf8f-3791-4b05-a9da-95ff4f275702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:47.808168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 965/1925] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:47.812479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b39a4f-fd82-4411-880d-bee5aed1e179 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:47.822581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b39a4f-fd82-4411-880d-bee5aed1e179 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:47.827117 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3e931-5490-4891-94ba-5636829a2af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:47.827797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3e931-5490-4891-94ba-5636829a2af0 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:47.828081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3e931-5490-4891-94ba-5636829a2af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:47.828382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3e931-5490-4891-94ba-5636829a2af0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:47.829100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3e931-5490-4891-94ba-5636829a2af0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:47.829433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3e931-5490-4891-94ba-5636829a2af0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:47.831636 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3e931-5490-4891-94ba-5636829a2af0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:47.843076 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 961/1926] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:47.852309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227cf8f-3791-4b05-a9da-95ff4f275702 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:47.861769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227cf8f-3791-4b05-a9da-95ff4f275702 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:47.866630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585c3c5-7544-45c1-aa58-45a12812dbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:47.867342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585c3c5-7544-45c1-aa58-45a12812dbb2 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:47.867627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585c3c5-7544-45c1-aa58-45a12812dbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:47.868024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585c3c5-7544-45c1-aa58-45a12812dbb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:47.868726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585c3c5-7544-45c1-aa58-45a12812dbb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:47.870649 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585c3c5-7544-45c1-aa58-45a12812dbb2 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:47.950655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 966/1927] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:47 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:47.954826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3e931-5490-4891-94ba-5636829a2af0 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:47.965742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3e931-5490-4891-94ba-5636829a2af0 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:48.021123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832987ed-991a-4431-b6e3-9e284296c314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:48.021795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832987ed-991a-4431-b6e3-9e284296c314 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:48.022347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832987ed-991a-4431-b6e3-9e284296c314 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:48.022619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832987ed-991a-4431-b6e3-9e284296c314 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:48.023304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832987ed-991a-4431-b6e3-9e284296c314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:48.025388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832987ed-991a-4431-b6e3-9e284296c314 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:48.125822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 962/1928] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:47 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:48.128231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585c3c5-7544-45c1-aa58-45a12812dbb2 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:48.140318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585c3c5-7544-45c1-aa58-45a12812dbb2 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:48.145370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6ba1-d134-40de-a453-b5a49e73054c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:48.146049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6ba1-d134-40de-a453-b5a49e73054c admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:48.146355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6ba1-d134-40de-a453-b5a49e73054c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:48.146681 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6ba1-d134-40de-a453-b5a49e73054c admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:48.147869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce6ba1-d134-40de-a453-b5a49e73054c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:48.149712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce6ba1-d134-40de-a453-b5a49e73054c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:48.162270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 967/1929] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:48.166580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832987ed-991a-4431-b6e3-9e284296c314 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:48.177514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832987ed-991a-4431-b6e3-9e284296c314 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:48.182582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c54c0-8afa-426a-a640-c182950a1b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:48.183273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c54c0-8afa-426a-a640-c182950a1b0d admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:48.183560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c54c0-8afa-426a-a640-c182950a1b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:48.183888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c54c0-8afa-426a-a640-c182950a1b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:48.184635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c54c0-8afa-426a-a640-c182950a1b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:48.186295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c54c0-8afa-426a-a640-c182950a1b0d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:48.197351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 963/1930] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:56:48.201725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6ba1-d134-40de-a453-b5a49e73054c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:48.211559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6ba1-d134-40de-a453-b5a49e73054c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:48.216478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233076ea-0a7e-40d9-b318-3e79ba5386bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:48.217362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233076ea-0a7e-40d9-b318-3e79ba5386bf admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:48.217646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233076ea-0a7e-40d9-b318-3e79ba5386bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:48.217916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233076ea-0a7e-40d9-b318-3e79ba5386bf admin admin] PATH_INFO: `/v3/projects/3eb8d4ed271249c0b90da1de7c4f88c3/users/d35e390ae2ad42a4991ae622442cb381/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:48.233372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233076ea-0a7e-40d9-b318-3e79ba5386bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb8d4ed271249c0b90da1de7c4f88c3, user_id=d35e390ae2ad42a4991ae622442cb381, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:48.236172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233076ea-0a7e-40d9-b318-3e79ba5386bf admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:48.311073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 968/1931] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:48 2022] PUT /identity/v3/projects/3eb8d4ed271249c0b90da1de7c4f88c3/users/d35e390ae2ad42a4991ae622442cb381/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:48.316951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f9411-e1bf-4549-8433-2e68f38fb453 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:48.317241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f9411-e1bf-4549-8433-2e68f38fb453 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:48.317516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f9411-e1bf-4549-8433-2e68f38fb453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:48.336620 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8f4f9411-e1bf-4549-8433-2e68f38fb453 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:48.337854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8f4f9411-e1bf-4549-8433-2e68f38fb453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:48.398244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 964/1932] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:48.408138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233076ea-0a7e-40d9-b318-3e79ba5386bf admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:48.410038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-233076ea-0a7e-40d9-b318-3e79ba5386bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:48.474463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233076ea-0a7e-40d9-b318-3e79ba5386bf admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:48.480810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86b4fc-08d5-4e70-89f0-c8bbd8a42512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:48.481631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86b4fc-08d5-4e70-89f0-c8bbd8a42512 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:48.481753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86b4fc-08d5-4e70-89f0-c8bbd8a42512 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:48.482188 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86b4fc-08d5-4e70-89f0-c8bbd8a42512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:48.495867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86b4fc-08d5-4e70-89f0-c8bbd8a42512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:48.497343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86b4fc-08d5-4e70-89f0-c8bbd8a42512 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:48.508299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 969/1933] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:48.535827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:49.381367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:49.713550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f9411-e1bf-4549-8433-2e68f38fb453 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:49.714927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8f4f9411-e1bf-4549-8433-2e68f38fb453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:49.726633 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f9411-e1bf-4549-8433-2e68f38fb453 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:49.731449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:49.732324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:49.732631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:49.732902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] PATH_INFO: `/v3/users/32031e7c80f04de1a6cce41a33d961e2` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:49.734225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] RBAC: Authorizing `identity:delete_user(user_id=32031e7c80f04de1a6cce41a33d961e2)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:49.737468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:50.080237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32031e7c80f04de1a6cce41a33d961e2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:50.081134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32031e7c80f04de1a6cce41a33d961e2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:50.094840 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32031e7c80f04de1a6cce41a33d961e2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:50.095577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32031e7c80f04de1a6cce41a33d961e2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:50.104031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32031e7c80f04de1a6cce41a33d961e2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:50.243829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 965/1934] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:49 2022] DELETE /identity/v3/users/32031e7c80f04de1a6cce41a33d961e2 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:50.248129 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe79920-d73f-43e1-b549-a25b71d61ba5 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:50.248611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe79920-d73f-43e1-b549-a25b71d61ba5 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:50.248705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe79920-d73f-43e1-b549-a25b71d61ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:50.270966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-5fe79920-d73f-43e1-b549-a25b71d61ba5 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:50.272756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5fe79920-d73f-43e1-b549-a25b71d61ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:50.354691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 970/1935] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:50.368050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:50.369872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:50.389957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553aa4e-e9b7-4867-9439-64996f80150a None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:50.393768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553aa4e-e9b7-4867-9439-64996f80150a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:50.394066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553aa4e-e9b7-4867-9439-64996f80150a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:50.428111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-6553aa4e-e9b7-4867-9439-64996f80150a None None] MFA Rules not processed for user `d35e390ae2ad42a4991ae622442cb381`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:50.429750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6553aa4e-e9b7-4867-9439-64996f80150a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:50.465198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086cdcc4-341f-4404-9489-5efbb802890b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:50.470020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efc7d6-36b0-4f02-b561-d84f6c513a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:50.470692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efc7d6-36b0-4f02-b561-d84f6c513a72 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:50.471023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efc7d6-36b0-4f02-b561-d84f6c513a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:50.471484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efc7d6-36b0-4f02-b561-d84f6c513a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:50.482565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2efc7d6-36b0-4f02-b561-d84f6c513a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:50.484303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2efc7d6-36b0-4f02-b561-d84f6c513a72 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:50.498625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 966/1936] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:50.506368 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 971/1937] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:50.510871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efc7d6-36b0-4f02-b561-d84f6c513a72 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:50.512475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a2efc7d6-36b0-4f02-b561-d84f6c513a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:50.526478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efc7d6-36b0-4f02-b561-d84f6c513a72 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:50.531630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30d62e-bcc3-4c06-8d7e-a9f74fafd9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:50.532306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30d62e-bcc3-4c06-8d7e-a9f74fafd9bd admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:50.533043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30d62e-bcc3-4c06-8d7e-a9f74fafd9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:50.533043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30d62e-bcc3-4c06-8d7e-a9f74fafd9bd admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:50.533913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30d62e-bcc3-4c06-8d7e-a9f74fafd9bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:50.535882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30d62e-bcc3-4c06-8d7e-a9f74fafd9bd admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:50.606732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:50.662922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 967/1938] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:50 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:50.667465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553aa4e-e9b7-4867-9439-64996f80150a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:50.679440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553aa4e-e9b7-4867-9439-64996f80150a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:50.793895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7d037-76e1-40d8-be57-12ebd75ef210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:50.794784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7d037-76e1-40d8-be57-12ebd75ef210 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:50.795377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7d037-76e1-40d8-be57-12ebd75ef210 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:50.795534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7d037-76e1-40d8-be57-12ebd75ef210 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:50.801286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7d037-76e1-40d8-be57-12ebd75ef210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:50.801286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7d037-76e1-40d8-be57-12ebd75ef210 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:50.895463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 972/1939] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:50 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:50.896166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30d62e-bcc3-4c06-8d7e-a9f74fafd9bd admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:50.914640 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30d62e-bcc3-4c06-8d7e-a9f74fafd9bd admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:50.928731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2cf94-9986-43e0-9e93-9d16d1ee3bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:50.929429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2cf94-9986-43e0-9e93-9d16d1ee3bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:50.929704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2cf94-9986-43e0-9e93-9d16d1ee3bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:50.930098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2cf94-9986-43e0-9e93-9d16d1ee3bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:50.930841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2cf94-9986-43e0-9e93-9d16d1ee3bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:50.932474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2cf94-9986-43e0-9e93-9d16d1ee3bd4 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:50.959554 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 968/1940] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:56:50.964332 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7d037-76e1-40d8-be57-12ebd75ef210 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:50.978898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7d037-76e1-40d8-be57-12ebd75ef210 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:50.986351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448afe4-782a-473c-9a56-d96f21a05e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:50.988802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448afe4-782a-473c-9a56-d96f21a05e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:50.989328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448afe4-782a-473c-9a56-d96f21a05e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:50.992089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448afe4-782a-473c-9a56-d96f21a05e9f admin admin] PATH_INFO: `/v3/projects/31ac93d3e19d41be9de01b85141bd17d/users/721d786a0c61430e8a51b8debe89e62b/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:51.026784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448afe4-782a-473c-9a56-d96f21a05e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ac93d3e19d41be9de01b85141bd17d, user_id=721d786a0c61430e8a51b8debe89e62b, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:51.033364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448afe4-782a-473c-9a56-d96f21a05e9f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:51.152241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 973/1941] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:50 2022] PUT /identity/v3/projects/31ac93d3e19d41be9de01b85141bd17d/users/721d786a0c61430e8a51b8debe89e62b/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:51.156231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2cf94-9986-43e0-9e93-9d16d1ee3bd4 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:51.168088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2cf94-9986-43e0-9e93-9d16d1ee3bd4 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:51.262321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac3ee3-6b05-4040-a30f-826352dfb7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:51.263688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac3ee3-6b05-4040-a30f-826352dfb7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:51.264151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac3ee3-6b05-4040-a30f-826352dfb7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:51.264669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac3ee3-6b05-4040-a30f-826352dfb7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:51.265641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac3ee3-6b05-4040-a30f-826352dfb7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:51.267468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac3ee3-6b05-4040-a30f-826352dfb7f8 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:51.281411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 969/1942] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:51.291371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448afe4-782a-473c-9a56-d96f21a05e9f admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:51.303858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448afe4-782a-473c-9a56-d96f21a05e9f admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:51.309777 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43903b7-b972-4c86-b342-610128a8affc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:51.309777 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43903b7-b972-4c86-b342-610128a8affc admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:51.310083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43903b7-b972-4c86-b342-610128a8affc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:51.310158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43903b7-b972-4c86-b342-610128a8affc admin admin] PATH_INFO: `/v3/projects/31ac93d3e19d41be9de01b85141bd17d/users/721d786a0c61430e8a51b8debe89e62b/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:51.312926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43903b7-b972-4c86-b342-610128a8affc admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ac93d3e19d41be9de01b85141bd17d, user_id=721d786a0c61430e8a51b8debe89e62b, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:51.315631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43903b7-b972-4c86-b342-610128a8affc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:51.328250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 974/1943] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:51 2022] PUT /identity/v3/projects/31ac93d3e19d41be9de01b85141bd17d/users/721d786a0c61430e8a51b8debe89e62b/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:51.341191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac3ee3-6b05-4040-a30f-826352dfb7f8 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:51.343101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0aac3ee3-6b05-4040-a30f-826352dfb7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:51.361061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac3ee3-6b05-4040-a30f-826352dfb7f8 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:51.430418 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:51.431355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:51.431642 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:51.431918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] PATH_INFO: `/v3/projects/49b30d8ba86242e7bce27d36af163443` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:51.433202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b30d8ba86242e7bce27d36af163443)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:51.435129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:51.573637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b30d8ba86242e7bce27d36af163443', 'request_id': 'req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:51.580012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b30d8ba86242e7bce27d36af163443', 'request_id': 'req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:51.588015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b30d8ba86242e7bce27d36af163443', 'request_id': 'req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:51.683520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] The token cache is being invalidate because project 49b30d8ba86242e7bce27d36af163443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:51.683620 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:51.684204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:51.686102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 970/1944] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:51 2022] DELETE /identity/v3/projects/49b30d8ba86242e7bce27d36af163443 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:51.691541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43903b7-b972-4c86-b342-610128a8affc admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:51.693287 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f43903b7-b972-4c86-b342-610128a8affc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:51.708997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43903b7-b972-4c86-b342-610128a8affc admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:51.791424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:51.792403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:51.792686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:51.792961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] PATH_INFO: `/v3/users/737ec2b8936740f9880eb05ca6c25ac1` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:51.794617 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=737ec2b8936740f9880eb05ca6c25ac1)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:51.796837 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:51.980390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:52.138763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '737ec2b8936740f9880eb05ca6c25ac1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:52.139477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '737ec2b8936740f9880eb05ca6c25ac1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:52.140103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '737ec2b8936740f9880eb05ca6c25ac1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:52.153411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '737ec2b8936740f9880eb05ca6c25ac1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:52.179769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '737ec2b8936740f9880eb05ca6c25ac1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:52.279119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 975/1945] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:51 2022] DELETE /identity/v3/users/737ec2b8936740f9880eb05ca6c25ac1 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:52.325652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:52.327141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:52.423440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71315a5-1573-49c9-9ff1-bf29bfbd9d96 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:52.471990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7bf73-aee2-405f-ac94-70069991c491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:52.472678 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7bf73-aee2-405f-ac94-70069991c491 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:52.472959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7bf73-aee2-405f-ac94-70069991c491 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:52.473268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7bf73-aee2-405f-ac94-70069991c491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:52.475270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c6e7bf73-aee2-405f-ac94-70069991c491 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:52.579469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7bf73-aee2-405f-ac94-70069991c491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:52.581060 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7bf73-aee2-405f-ac94-70069991c491 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:52.597158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 971/1946] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:52.856216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:52.857863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:52.953018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f160c-0a73-4784-82eb-9054d2ac4e90 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:52.959250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:52.960239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:52.960670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:52.961839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] PATH_INFO: `/v3/projects/3b885a22765d41fea6392f3caf1cf65c` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:52.963867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b885a22765d41fea6392f3caf1cf65c)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:52.966150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:53.137744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b885a22765d41fea6392f3caf1cf65c', 'request_id': 'req-4155611d-3175-4c77-b6ab-a6238631fbfc'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:53.149262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b885a22765d41fea6392f3caf1cf65c', 'request_id': 'req-4155611d-3175-4c77-b6ab-a6238631fbfc'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:53.156076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b885a22765d41fea6392f3caf1cf65c', 'request_id': 'req-4155611d-3175-4c77-b6ab-a6238631fbfc'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:53.300469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] The token cache is being invalidate because project 3b885a22765d41fea6392f3caf1cf65c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:53.300840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:53.301582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4155611d-3175-4c77-b6ab-a6238631fbfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:53.303511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 976/1947] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:52 2022] DELETE /identity/v3/projects/3b885a22765d41fea6392f3caf1cf65c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:53.309208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf4324-e2f4-4713-9aa3-88370e5f6900 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:53.309208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf4324-e2f4-4713-9aa3-88370e5f6900 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:53.309475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf4324-e2f4-4713-9aa3-88370e5f6900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:53.336534 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-cdcf4324-e2f4-4713-9aa3-88370e5f6900 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:53.339166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-cdcf4324-e2f4-4713-9aa3-88370e5f6900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:53.410239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 972/1948] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:53.418311 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b7fa9-6d2f-48bb-9eb7-2e9a131dc29e None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:53.418614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b7fa9-6d2f-48bb-9eb7-2e9a131dc29e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:53.419132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b7fa9-6d2f-48bb-9eb7-2e9a131dc29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:53.455673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0a2b7fa9-6d2f-48bb-9eb7-2e9a131dc29e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:53.457286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0a2b7fa9-6d2f-48bb-9eb7-2e9a131dc29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:53.465694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 977/1949] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:53.470444 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf4324-e2f4-4713-9aa3-88370e5f6900 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:53.483703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf4324-e2f4-4713-9aa3-88370e5f6900 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:53.488323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:53.490986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf730e-18b1-4096-9f02-065d7e0887f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:53.491927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf730e-18b1-4096-9f02-065d7e0887f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:53.492313 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf730e-18b1-4096-9f02-065d7e0887f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:53.493806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf730e-18b1-4096-9f02-065d7e0887f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:53.494181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf730e-18b1-4096-9f02-065d7e0887f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:53.494759 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf730e-18b1-4096-9f02-065d7e0887f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:53.497827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebf730e-18b1-4096-9f02-065d7e0887f3 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:53.524665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 973/1950] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:53.530013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b7fa9-6d2f-48bb-9eb7-2e9a131dc29e None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:53.540969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b7fa9-6d2f-48bb-9eb7-2e9a131dc29e None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:53.546297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733605da-a90b-467f-8616-9d81a874bd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:53.547184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733605da-a90b-467f-8616-9d81a874bd31 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:53.547766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733605da-a90b-467f-8616-9d81a874bd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:53.548497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733605da-a90b-467f-8616-9d81a874bd31 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:53.548940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733605da-a90b-467f-8616-9d81a874bd31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:53.551525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733605da-a90b-467f-8616-9d81a874bd31 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:53.600886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf730e-18b1-4096-9f02-065d7e0887f3 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:53.603474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-bebf730e-18b1-4096-9f02-065d7e0887f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:53.620865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf730e-18b1-4096-9f02-065d7e0887f3 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:53.625671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:53.626842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:53.627130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:53.627403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] PATH_INFO: `/v3/users/973dd100274c46be8f13365b3ba8c971` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:53.629385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] RBAC: Authorizing `identity:delete_user(user_id=973dd100274c46be8f13365b3ba8c971)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:53.631475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:53.665444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 978/1951] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:53 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:53.669270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733605da-a90b-467f-8616-9d81a874bd31 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:53.728791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:53.743763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733605da-a90b-467f-8616-9d81a874bd31 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:53.749036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92cfd0-9410-4d5c-93c9-72514c769df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:53.749774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92cfd0-9410-4d5c-93c9-72514c769df2 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:53.750073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92cfd0-9410-4d5c-93c9-72514c769df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:53.750356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92cfd0-9410-4d5c-93c9-72514c769df2 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:53.751114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92cfd0-9410-4d5c-93c9-72514c769df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:53.753838 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92cfd0-9410-4d5c-93c9-72514c769df2 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:53.872616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 979/1952] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:53 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:53.876954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92cfd0-9410-4d5c-93c9-72514c769df2 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:53.890316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '973dd100274c46be8f13365b3ba8c971'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:53.891010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '973dd100274c46be8f13365b3ba8c971'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:53.896838 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92cfd0-9410-4d5c-93c9-72514c769df2 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:53.903084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658a0ce-c08b-477f-9623-18784345bbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:53.903297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658a0ce-c08b-477f-9623-18784345bbba admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:53.903705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658a0ce-c08b-477f-9623-18784345bbba admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:53.904054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658a0ce-c08b-477f-9623-18784345bbba admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:53.904921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658a0ce-c08b-477f-9623-18784345bbba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:53.906456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '973dd100274c46be8f13365b3ba8c971'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:53.906945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658a0ce-c08b-477f-9623-18784345bbba admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:53.907242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '973dd100274c46be8f13365b3ba8c971'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:53.916974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '973dd100274c46be8f13365b3ba8c971'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:53.917923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 980/1953] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:53.919298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ce0f1-1950-4ca5-bda7-1df49347ae10 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:53.919597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ce0f1-1950-4ca5-bda7-1df49347ae10 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:53.919876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ce0f1-1950-4ca5-bda7-1df49347ae10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:53.949183 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-405ce0f1-1950-4ca5-bda7-1df49347ae10 None None] MFA Rules not processed for user `721d786a0c61430e8a51b8debe89e62b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:53.950491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-405ce0f1-1950-4ca5-bda7-1df49347ae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:54.012016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 974/1954] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:53 2022] DELETE /identity/v3/users/973dd100274c46be8f13365b3ba8c971 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:54.015872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:54.017250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:54.022077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 981/1955] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:54.022077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b84d4-5c1f-4a0c-af0d-65d40c97f2f5 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:54.022077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b84d4-5c1f-4a0c-af0d-65d40c97f2f5 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:54.022077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b84d4-5c1f-4a0c-af0d-65d40c97f2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:54.045300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-742b84d4-5c1f-4a0c-af0d-65d40c97f2f5 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:54.046508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-742b84d4-5c1f-4a0c-af0d-65d40c97f2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:54.090766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb952cc1-18fa-4f7c-a4db-fd4d4b62d22d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:54.095538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf23f6f-f89f-41bb-b271-8f4b524f63f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:54.096236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf23f6f-f89f-41bb-b271-8f4b524f63f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:54.096556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf23f6f-f89f-41bb-b271-8f4b524f63f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:54.096868 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf23f6f-f89f-41bb-b271-8f4b524f63f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:54.097565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf23f6f-f89f-41bb-b271-8f4b524f63f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:54.099167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf23f6f-f89f-41bb-b271-8f4b524f63f5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:54.106352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 982/1956] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:54.107477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b84d4-5c1f-4a0c-af0d-65d40c97f2f5 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:54.109032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-742b84d4-5c1f-4a0c-af0d-65d40c97f2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:54.113607 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 975/1957] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:54.114924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf23f6f-f89f-41bb-b271-8f4b524f63f5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:54.124789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf23f6f-f89f-41bb-b271-8f4b524f63f5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:54.131199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575292e-c5a3-48e0-bac0-f7544796bf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:54.136338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575292e-c5a3-48e0-bac0-f7544796bf7d admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:54.136338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575292e-c5a3-48e0-bac0-f7544796bf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:54.136338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575292e-c5a3-48e0-bac0-f7544796bf7d admin admin] PATH_INFO: `/v3/projects/54c017ff390041e6845811b7714be64b/users/13e5a6dceaca463b8ab768ca630306c9/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:54.148845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575292e-c5a3-48e0-bac0-f7544796bf7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c017ff390041e6845811b7714be64b, user_id=13e5a6dceaca463b8ab768ca630306c9, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:54.151512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575292e-c5a3-48e0-bac0-f7544796bf7d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:54.172751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b84d4-5c1f-4a0c-af0d-65d40c97f2f5 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:54.177194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c19e0-3c12-4288-936e-d404869fd7f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:54.177852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c19e0-3c12-4288-936e-d404869fd7f1 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:54.178137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c19e0-3c12-4288-936e-d404869fd7f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:54.178417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c19e0-3c12-4288-936e-d404869fd7f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:54.180217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-de0c19e0-3c12-4288-936e-d404869fd7f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:54.239381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c19e0-3c12-4288-936e-d404869fd7f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:54.240895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c19e0-3c12-4288-936e-d404869fd7f1 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:54.250454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 983/1958] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:54.251323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c19e0-3c12-4288-936e-d404869fd7f1 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:54.254026 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-de0c19e0-3c12-4288-936e-d404869fd7f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:54.264069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 976/1959] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:54 2022] PUT /identity/v3/projects/54c017ff390041e6845811b7714be64b/users/13e5a6dceaca463b8ab768ca630306c9/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:54.275672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df5fe1-da49-464f-afca-de7c58634c40 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:54.275958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df5fe1-da49-464f-afca-de7c58634c40 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:54.276242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df5fe1-da49-464f-afca-de7c58634c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:54.304120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-64df5fe1-da49-464f-afca-de7c58634c40 None None] MFA Rules not processed for user `13e5a6dceaca463b8ab768ca630306c9`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:54.305417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-64df5fe1-da49-464f-afca-de7c58634c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:54.322528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c19e0-3c12-4288-936e-d404869fd7f1 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:54.374067 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ba2e0-5922-4fcd-9371-89345ea7d24c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:54.374723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ba2e0-5922-4fcd-9371-89345ea7d24c service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:54.375003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ba2e0-5922-4fcd-9371-89345ea7d24c service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:54.375277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ba2e0-5922-4fcd-9371-89345ea7d24c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:54.381403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 977/1960] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:54.386236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df5fe1-da49-464f-afca-de7c58634c40 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:54.454905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ba2e0-5922-4fcd-9371-89345ea7d24c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:54.455044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df5fe1-da49-464f-afca-de7c58634c40 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:54.457779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ba2e0-5922-4fcd-9371-89345ea7d24c service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:54.459911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a2daa-f4d9-47b2-8506-ac16c29a69da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:54.460639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a2daa-f4d9-47b2-8506-ac16c29a69da admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:54.460952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a2daa-f4d9-47b2-8506-ac16c29a69da admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:54.461191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a2daa-f4d9-47b2-8506-ac16c29a69da admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:54.461886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a2daa-f4d9-47b2-8506-ac16c29a69da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:54.463842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a2daa-f4d9-47b2-8506-ac16c29a69da admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:54.473402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 984/1961] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:54.571215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 978/1962] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:54 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:56:54.575675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ba2e0-5922-4fcd-9371-89345ea7d24c service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:54.585646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ba2e0-5922-4fcd-9371-89345ea7d24c service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:54.659829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf96e8a-93dc-4865-a5d6-a70252ef9999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:54.659829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf96e8a-93dc-4865-a5d6-a70252ef9999 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:54.659829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf96e8a-93dc-4865-a5d6-a70252ef9999 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:54.659829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf96e8a-93dc-4865-a5d6-a70252ef9999 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:54.660221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf96e8a-93dc-4865-a5d6-a70252ef9999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:54.660221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf96e8a-93dc-4865-a5d6-a70252ef9999 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:54.753767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 985/1963] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:54 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:56:54.756676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a2daa-f4d9-47b2-8506-ac16c29a69da admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:54.767665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a2daa-f4d9-47b2-8506-ac16c29a69da admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:54.773231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a99818-c7a0-40f7-a3ab-08efdde71cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:54.787627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a99818-c7a0-40f7-a3ab-08efdde71cfa admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:54.787627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a99818-c7a0-40f7-a3ab-08efdde71cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:54.787627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a99818-c7a0-40f7-a3ab-08efdde71cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:54.787627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a99818-c7a0-40f7-a3ab-08efdde71cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:54.787627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a99818-c7a0-40f7-a3ab-08efdde71cfa admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:54.802933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 979/1964] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:54.806933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf96e8a-93dc-4865-a5d6-a70252ef9999 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:54.822487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf96e8a-93dc-4865-a5d6-a70252ef9999 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:54.823566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a34c48-5306-4fba-b15f-c47c882a0765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:54.824605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a34c48-5306-4fba-b15f-c47c882a0765 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:54.824892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a34c48-5306-4fba-b15f-c47c882a0765 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:54.825170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a34c48-5306-4fba-b15f-c47c882a0765 admin admin] PATH_INFO: `/v3/projects/c2ec7003d5f444298214b5370f94c0d3/users/5e49b84999fc4685a04e7210f8ff3467/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:54.843152 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a34c48-5306-4fba-b15f-c47c882a0765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ec7003d5f444298214b5370f94c0d3, user_id=5e49b84999fc4685a04e7210f8ff3467, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:54.845860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a34c48-5306-4fba-b15f-c47c882a0765 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:55.002715 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 986/1965] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:54 2022] PUT /identity/v3/projects/c2ec7003d5f444298214b5370f94c0d3/users/5e49b84999fc4685a04e7210f8ff3467/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:55.006876 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a99818-c7a0-40f7-a3ab-08efdde71cfa admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:55.016928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a99818-c7a0-40f7-a3ab-08efdde71cfa admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:55.087471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4815522-480f-41f2-9adf-92c2b4b6fb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:55.088147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4815522-480f-41f2-9adf-92c2b4b6fb3c admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:55.088452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4815522-480f-41f2-9adf-92c2b4b6fb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:55.088729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4815522-480f-41f2-9adf-92c2b4b6fb3c admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:55.089461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4815522-480f-41f2-9adf-92c2b4b6fb3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:55.091046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4815522-480f-41f2-9adf-92c2b4b6fb3c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:55.103982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a34c48-5306-4fba-b15f-c47c882a0765 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:55.105857 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-42a34c48-5306-4fba-b15f-c47c882a0765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:55.110196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 980/1966] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:55.116182 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4815522-480f-41f2-9adf-92c2b4b6fb3c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:55.120414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a34c48-5306-4fba-b15f-c47c882a0765 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:55.124606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:55.125486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:55.125797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:55.126073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] PATH_INFO: `/v3/projects/1cf850896a0b4d00a895599b1d515b96` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:55.127278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf850896a0b4d00a895599b1d515b96)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:55.129434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:55.142610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4815522-480f-41f2-9adf-92c2b4b6fb3c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:55.151358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796c2b0-abaf-4830-bef1-ebd3a54a9113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:55.151358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796c2b0-abaf-4830-bef1-ebd3a54a9113 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:55.151358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796c2b0-abaf-4830-bef1-ebd3a54a9113 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:55.151358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796c2b0-abaf-4830-bef1-ebd3a54a9113 admin admin] PATH_INFO: `/v3/projects/c2ec7003d5f444298214b5370f94c0d3/users/5e49b84999fc4685a04e7210f8ff3467/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:55.153364 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c796c2b0-abaf-4830-bef1-ebd3a54a9113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ec7003d5f444298214b5370f94c0d3, user_id=5e49b84999fc4685a04e7210f8ff3467, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:55.156177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c796c2b0-abaf-4830-bef1-ebd3a54a9113 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:55.168486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 981/1967] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:55 2022] PUT /identity/v3/projects/c2ec7003d5f444298214b5370f94c0d3/users/5e49b84999fc4685a04e7210f8ff3467/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:55.178458 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163b09-9681-49e4-8381-5bcb2da451f7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:55.178899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163b09-9681-49e4-8381-5bcb2da451f7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:55.179302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163b09-9681-49e4-8381-5bcb2da451f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:55.219935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-15163b09-9681-49e4-8381-5bcb2da451f7 None None] MFA Rules not processed for user `5e49b84999fc4685a04e7210f8ff3467`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:55.221204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-15163b09-9681-49e4-8381-5bcb2da451f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:55.299384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 982/1968] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:55.302057 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf850896a0b4d00a895599b1d515b96', 'request_id': 'req-cc484eca-f06e-4bf1-8c5d-c09709701a5e'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:55.310114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf850896a0b4d00a895599b1d515b96', 'request_id': 'req-cc484eca-f06e-4bf1-8c5d-c09709701a5e'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:55.312653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15163b09-9681-49e4-8381-5bcb2da451f7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:55.314315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-15163b09-9681-49e4-8381-5bcb2da451f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:55.319504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf850896a0b4d00a895599b1d515b96', 'request_id': 'req-cc484eca-f06e-4bf1-8c5d-c09709701a5e'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:55.378770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15163b09-9681-49e4-8381-5bcb2da451f7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:55.419337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] The token cache is being invalidate because project 1cf850896a0b4d00a895599b1d515b96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:55.419630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:55.420290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:55.421884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 987/1969] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:55 2022] DELETE /identity/v3/projects/1cf850896a0b4d00a895599b1d515b96 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:55.426064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:55.427454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:55.430425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c6b8f-f1aa-4742-af52-1240beb0067e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:55.431082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c6b8f-f1aa-4742-af52-1240beb0067e service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:55.431368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c6b8f-f1aa-4742-af52-1240beb0067e service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:55.431647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c6b8f-f1aa-4742-af52-1240beb0067e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:55.433580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-343c6b8f-f1aa-4742-af52-1240beb0067e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:55.495295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc484eca-f06e-4bf1-8c5d-c09709701a5e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:55.499987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:55.500892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:55.501174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:55.501499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] PATH_INFO: `/v3/users/32c9bd0f8e6b4d9caffc7b766dc406be` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:55.503209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c9bd0f8e6b4d9caffc7b766dc406be)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:55.505160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:55.507684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c6b8f-f1aa-4742-af52-1240beb0067e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:55.509148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c6b8f-f1aa-4742-af52-1240beb0067e service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:55.520252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 983/1970] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:55.651669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:55.665384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c9bd0f8e6b4d9caffc7b766dc406be'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:55.666966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c9bd0f8e6b4d9caffc7b766dc406be'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:55.667656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c9bd0f8e6b4d9caffc7b766dc406be'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:55.677664 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c9bd0f8e6b4d9caffc7b766dc406be'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:55.692132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c9bd0f8e6b4d9caffc7b766dc406be'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:55.817844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 988/1971] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:55 2022] DELETE /identity/v3/users/32c9bd0f8e6b4d9caffc7b766dc406be => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:56.099627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:56.187105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c6b8f-f1aa-4742-af52-1240beb0067e service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:56.188588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-343c6b8f-f1aa-4742-af52-1240beb0067e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:56.200877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:56.202357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:56.204145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c6b8f-f1aa-4742-af52-1240beb0067e service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:56.266283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:56.267234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:56.267519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:56.267847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] PATH_INFO: `/v3/users/13e5a6dceaca463b8ab768ca630306c9` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:56.269317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e5a6dceaca463b8ab768ca630306c9)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:56.269629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc8cff-a566-48d8-bd06-45b077ece5e5 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:56.271344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:56.273602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a0ebf-6094-4f14-8970-f74fb902dca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:56.274275 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a0ebf-6094-4f14-8970-f74fb902dca1 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:56.274523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a0ebf-6094-4f14-8970-f74fb902dca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:56.274782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a0ebf-6094-4f14-8970-f74fb902dca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:56.276692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-273a0ebf-6094-4f14-8970-f74fb902dca1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:56.356658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a0ebf-6094-4f14-8970-f74fb902dca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:56.358122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a0ebf-6094-4f14-8970-f74fb902dca1 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:56.371708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 989/1972] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:56:56.372816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a0ebf-6094-4f14-8970-f74fb902dca1 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:56.373901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-273a0ebf-6094-4f14-8970-f74fb902dca1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:56.387726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a0ebf-6094-4f14-8970-f74fb902dca1 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:56.393785 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:56.395036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:56.395036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:56.395393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] PATH_INFO: `/v3/projects/f22d7c545b5a4241b309496095e6134f` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:56.396568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f22d7c545b5a4241b309496095e6134f)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:56.399712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:56.479509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e5a6dceaca463b8ab768ca630306c9'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:56.480211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e5a6dceaca463b8ab768ca630306c9'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:56.491796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e5a6dceaca463b8ab768ca630306c9'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:56.492476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e5a6dceaca463b8ab768ca630306c9'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:56.499749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-93f0448f-2238-4dae-9c4c-f92dfe6dd208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e5a6dceaca463b8ab768ca630306c9'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:56.519334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f22d7c545b5a4241b309496095e6134f', 'request_id': 'req-afa59445-3c13-4700-820c-f40ad0f0c052'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:56.525730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f22d7c545b5a4241b309496095e6134f', 'request_id': 'req-afa59445-3c13-4700-820c-f40ad0f0c052'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:56.531243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f22d7c545b5a4241b309496095e6134f', 'request_id': 'req-afa59445-3c13-4700-820c-f40ad0f0c052'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:56.591955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 984/1973] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:56 2022] DELETE /identity/v3/users/13e5a6dceaca463b8ab768ca630306c9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:56.598261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605db06-5dcd-42df-abb9-f9d3e6007a81 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:56.598552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605db06-5dcd-42df-abb9-f9d3e6007a81 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:56.598827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605db06-5dcd-42df-abb9-f9d3e6007a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:56.617172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9605db06-5dcd-42df-abb9-f9d3e6007a81 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:56.618490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9605db06-5dcd-42df-abb9-f9d3e6007a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:56.648022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] The token cache is being invalidate because project f22d7c545b5a4241b309496095e6134f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:56.649600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:56.649921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-afa59445-3c13-4700-820c-f40ad0f0c052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:56.652088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 990/1974] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:56 2022] DELETE /identity/v3/projects/f22d7c545b5a4241b309496095e6134f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:56.656744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259c8b9-8716-4867-891b-73262751d29b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:56.657154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259c8b9-8716-4867-891b-73262751d29b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:56.657520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259c8b9-8716-4867-891b-73262751d29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:56.682650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-1259c8b9-8716-4867-891b-73262751d29b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:56.684018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1259c8b9-8716-4867-891b-73262751d29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:56.692432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 991/1975] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:56.693338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 985/1976] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:56.702545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc563a6a-d811-4dfb-867c-79750945dc4f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:56.702834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc563a6a-d811-4dfb-867c-79750945dc4f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:56.703126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc563a6a-d811-4dfb-867c-79750945dc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:56.705149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605db06-5dcd-42df-abb9-f9d3e6007a81 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:56.707247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9605db06-5dcd-42df-abb9-f9d3e6007a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:56.725523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-bc563a6a-d811-4dfb-867c-79750945dc4f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:56.726725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bc563a6a-d811-4dfb-867c-79750945dc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:56.735093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 992/1977] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:56.738994 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc563a6a-d811-4dfb-867c-79750945dc4f None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:56.747983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc563a6a-d811-4dfb-867c-79750945dc4f None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:56.752818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e3667-a37e-423d-bfef-3f80102cf53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:56.753914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e3667-a37e-423d-bfef-3f80102cf53c admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:56.757943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e3667-a37e-423d-bfef-3f80102cf53c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:56.757943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e3667-a37e-423d-bfef-3f80102cf53c admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:56.757943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e3667-a37e-423d-bfef-3f80102cf53c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:56.757943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e3667-a37e-423d-bfef-3f80102cf53c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:56.757943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e3667-a37e-423d-bfef-3f80102cf53c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:56.774481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 993/1978] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:56:56.775500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e3667-a37e-423d-bfef-3f80102cf53c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:56.780509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605db06-5dcd-42df-abb9-f9d3e6007a81 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:56.785173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e8acf-0cb4-493a-8e39-20745a1a0ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:56.785968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8acf-0cb4-493a-8e39-20745a1a0ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:56.786508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e3667-a37e-423d-bfef-3f80102cf53c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:56.786610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8acf-0cb4-493a-8e39-20745a1a0ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:56.786847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8acf-0cb4-493a-8e39-20745a1a0ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:56.791720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fe3ff-f35b-4dd0-aa78-638a939057c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:56.792405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fe3ff-f35b-4dd0-aa78-638a939057c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:56.792812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fe3ff-f35b-4dd0-aa78-638a939057c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:56.792964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fe3ff-f35b-4dd0-aa78-638a939057c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:56.793709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fe3ff-f35b-4dd0-aa78-638a939057c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:56.795648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fe3ff-f35b-4dd0-aa78-638a939057c6 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:56.797455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e8acf-0cb4-493a-8e39-20745a1a0ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:56.799063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e8acf-0cb4-493a-8e39-20745a1a0ca7 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:56.809161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 986/1979] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:56.907289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 994/1980] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:56 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:56:56.909305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e8acf-0cb4-493a-8e39-20745a1a0ca7 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:56.921728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e8acf-0cb4-493a-8e39-20745a1a0ca7 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:56.980386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:56.986444 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12486bf8-3e67-4fc4-8087-d8cb762b99fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:56.987155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12486bf8-3e67-4fc4-8087-d8cb762b99fe admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:56.987442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12486bf8-3e67-4fc4-8087-d8cb762b99fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:56.987723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12486bf8-3e67-4fc4-8087-d8cb762b99fe admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:56.988489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12486bf8-3e67-4fc4-8087-d8cb762b99fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:56.990403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12486bf8-3e67-4fc4-8087-d8cb762b99fe admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:57.090896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 987/1981] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:56 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:57.095085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fe3ff-f35b-4dd0-aa78-638a939057c6 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:57.106107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fe3ff-f35b-4dd0-aa78-638a939057c6 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:57.111491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80a3e9-af1a-46bd-9aaa-f98c1fcfe543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:57.112454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80a3e9-af1a-46bd-9aaa-f98c1fcfe543 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:57.112729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80a3e9-af1a-46bd-9aaa-f98c1fcfe543 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:57.113020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80a3e9-af1a-46bd-9aaa-f98c1fcfe543 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:57.113747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80a3e9-af1a-46bd-9aaa-f98c1fcfe543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:57.115414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80a3e9-af1a-46bd-9aaa-f98c1fcfe543 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:57.126620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 995/1982] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:57.131604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12486bf8-3e67-4fc4-8087-d8cb762b99fe admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:57.145845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12486bf8-3e67-4fc4-8087-d8cb762b99fe admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:57.152283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a845086-4e54-461f-9053-0ef4dc5622c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:57.153489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a845086-4e54-461f-9053-0ef4dc5622c9 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:57.154169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a845086-4e54-461f-9053-0ef4dc5622c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:57.154550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a845086-4e54-461f-9053-0ef4dc5622c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:57.155399 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a845086-4e54-461f-9053-0ef4dc5622c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:57.157125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a845086-4e54-461f-9053-0ef4dc5622c9 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:57.170395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 988/1983] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:56:57.171789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:57.174412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80a3e9-af1a-46bd-9aaa-f98c1fcfe543 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:57.188968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80a3e9-af1a-46bd-9aaa-f98c1fcfe543 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:57.195432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dcacd-2ee7-417f-b21b-2a16e2c1598d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:57.196347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dcacd-2ee7-417f-b21b-2a16e2c1598d admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:57.197031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dcacd-2ee7-417f-b21b-2a16e2c1598d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:57.197432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dcacd-2ee7-417f-b21b-2a16e2c1598d admin admin] PATH_INFO: `/v3/projects/25c8d1a9a0d64933bdcd65ee2edda27a/users/e3a04b3daf834ff78f085f3c93099c62/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:57.210008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a845086-4e54-461f-9053-0ef4dc5622c9 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:57.213451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2a845086-4e54-461f-9053-0ef4dc5622c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:57.216050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915dcacd-2ee7-417f-b21b-2a16e2c1598d admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c8d1a9a0d64933bdcd65ee2edda27a, user_id=e3a04b3daf834ff78f085f3c93099c62, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:57.220565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915dcacd-2ee7-417f-b21b-2a16e2c1598d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:57.231582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a845086-4e54-461f-9053-0ef4dc5622c9 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:57.236638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:57.238121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:57.238470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:57.238832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] PATH_INFO: `/v3/users/a0f41b54baa44d528cc6723be16dc01c` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:57.240192 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f41b54baa44d528cc6723be16dc01c)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:57.242262 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:57.317043 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 996/1984] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:56:57 2022] PUT /identity/v3/projects/25c8d1a9a0d64933bdcd65ee2edda27a/users/e3a04b3daf834ff78f085f3c93099c62/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:57.328472 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb0edb-bc55-4955-a77c-4731f50e271c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:57.328772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb0edb-bc55-4955-a77c-4731f50e271c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:57.329054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb0edb-bc55-4955-a77c-4731f50e271c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:57.362596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-85bb0edb-bc55-4955-a77c-4731f50e271c None None] MFA Rules not processed for user `e3a04b3daf834ff78f085f3c93099c62`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:57.363896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-85bb0edb-bc55-4955-a77c-4731f50e271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:57.430745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 997/1985] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:56:57.431673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb0edb-bc55-4955-a77c-4731f50e271c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:57.433460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-85bb0edb-bc55-4955-a77c-4731f50e271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:57.448956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f41b54baa44d528cc6723be16dc01c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.449703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f41b54baa44d528cc6723be16dc01c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.469418 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f41b54baa44d528cc6723be16dc01c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.470388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f41b54baa44d528cc6723be16dc01c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.481460 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f41b54baa44d528cc6723be16dc01c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.507715 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb0edb-bc55-4955-a77c-4731f50e271c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:57.512421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:57.513283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:57.513569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:57.513850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] PATH_INFO: `/v3/projects/54c017ff390041e6845811b7714be64b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:57.515044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c017ff390041e6845811b7714be64b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:57.517007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:57.557123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 989/1986] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:57 2022] DELETE /identity/v3/users/a0f41b54baa44d528cc6723be16dc01c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:57.557978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:57.559346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:57.612881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c017ff390041e6845811b7714be64b', 'request_id': 'req-e8a03747-bef4-49f7-ab91-fae17ef30a41'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.615884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd6484-b257-45a8-850c-1e5ebae8630f admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:57.619068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c017ff390041e6845811b7714be64b', 'request_id': 'req-e8a03747-bef4-49f7-ab91-fae17ef30a41'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.624814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c017ff390041e6845811b7714be64b', 'request_id': 'req-e8a03747-bef4-49f7-ab91-fae17ef30a41'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.664717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4d8ab-933a-47e0-a0b3-505af596245b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:57.665374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4d8ab-933a-47e0-a0b3-505af596245b service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:57.665681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4d8ab-933a-47e0-a0b3-505af596245b service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:57.668946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4d8ab-933a-47e0-a0b3-505af596245b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:57.668946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-88c4d8ab-933a-47e0-a0b3-505af596245b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:57.699779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] The token cache is being invalidate because project 54c017ff390041e6845811b7714be64b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:57.700130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.700811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e8a03747-bef4-49f7-ab91-fae17ef30a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.703812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 998/1987] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:57 2022] DELETE /identity/v3/projects/54c017ff390041e6845811b7714be64b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:57.703812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326fef9-a04a-43c7-a818-e84393ad5365 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:57.703969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326fef9-a04a-43c7-a818-e84393ad5365 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:57.704150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326fef9-a04a-43c7-a818-e84393ad5365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:57.722137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2326fef9-a04a-43c7-a818-e84393ad5365 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:57.723369 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2326fef9-a04a-43c7-a818-e84393ad5365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:57.730761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4d8ab-933a-47e0-a0b3-505af596245b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:57.732276 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c4d8ab-933a-47e0-a0b3-505af596245b service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:57.741184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 990/1988] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:57.742056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4d8ab-933a-47e0-a0b3-505af596245b service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:57.743434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-88c4d8ab-933a-47e0-a0b3-505af596245b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:57.790112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 999/1989] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:57.791037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326fef9-a04a-43c7-a818-e84393ad5365 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:57.792331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2326fef9-a04a-43c7-a818-e84393ad5365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:57.804957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326fef9-a04a-43c7-a818-e84393ad5365 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:57.809704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:57.810575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:57.810875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:57.811132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] PATH_INFO: `/v3/users/e3a04b3daf834ff78f085f3c93099c62` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:57.811326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4d8ab-933a-47e0-a0b3-505af596245b service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:57.812382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a04b3daf834ff78f085f3c93099c62)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:57.814303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:57.816639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:57.817519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:57.817798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:57.818070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] PATH_INFO: `/v3/users/5e49b84999fc4685a04e7210f8ff3467` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:57.819386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e49b84999fc4685a04e7210f8ff3467)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:57.821311 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:57.974918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a04b3daf834ff78f085f3c93099c62'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.975584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a04b3daf834ff78f085f3c93099c62'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:57.977931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a04b3daf834ff78f085f3c93099c62'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:58.001071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a04b3daf834ff78f085f3c93099c62'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:58.014376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a04b3daf834ff78f085f3c93099c62'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:58.023639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:58.120025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e49b84999fc4685a04e7210f8ff3467'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:58.120025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e49b84999fc4685a04e7210f8ff3467'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:58.134760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e49b84999fc4685a04e7210f8ff3467'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:58.135710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e49b84999fc4685a04e7210f8ff3467'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:58.147708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8c03ce08-2fda-48b8-a13c-27cd7d57f697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e49b84999fc4685a04e7210f8ff3467'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:58.153048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1000/1990] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:57 2022] DELETE /identity/v3/users/e3a04b3daf834ff78f085f3c93099c62 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:58.157238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:58.158722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:58.230821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3988c3-ce2d-46a2-b9ac-d721056782e3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:58.234126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:58.236010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6ea3-82c2-42cd-b5fc-b22af214b3eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:58.236779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6ea3-82c2-42cd-b5fc-b22af214b3eb service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:58.248143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6ea3-82c2-42cd-b5fc-b22af214b3eb service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:58.248143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6ea3-82c2-42cd-b5fc-b22af214b3eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:58.248143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-512b6ea3-82c2-42cd-b5fc-b22af214b3eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:58.277667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 991/1991] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:57 2022] DELETE /identity/v3/users/5e49b84999fc4685a04e7210f8ff3467 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:58.284908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424145c-c822-49dc-9738-d255f21c8ad6 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:58.284908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424145c-c822-49dc-9738-d255f21c8ad6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:58.284908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424145c-c822-49dc-9738-d255f21c8ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:58.300763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-0424145c-c822-49dc-9738-d255f21c8ad6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:58.302029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0424145c-c822-49dc-9738-d255f21c8ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:58.324908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b6ea3-82c2-42cd-b5fc-b22af214b3eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:58.326714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b6ea3-82c2-42cd-b5fc-b22af214b3eb service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:58.338971 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1001/1992] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:58.369737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 992/1993] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:58.379244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6ea3-82c2-42cd-b5fc-b22af214b3eb service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:58.380835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-512b6ea3-82c2-42cd-b5fc-b22af214b3eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:58.392411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6ea3-82c2-42cd-b5fc-b22af214b3eb service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:58.448058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5f7f5-9ea8-4559-937b-118d8f878e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:58.448754 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5f7f5-9ea8-4559-937b-118d8f878e0b service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:58.449088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5f7f5-9ea8-4559-937b-118d8f878e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:58.449416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5f7f5-9ea8-4559-937b-118d8f878e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:58.462512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5f7f5-9ea8-4559-937b-118d8f878e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:58.464091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5f7f5-9ea8-4559-937b-118d8f878e0b service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:58.474630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1002/1994] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:56:58.857692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424145c-c822-49dc-9738-d255f21c8ad6 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:58.859347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0424145c-c822-49dc-9738-d255f21c8ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:58.888032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5f7f5-9ea8-4559-937b-118d8f878e0b service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:58.889582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e7c5f7f5-9ea8-4559-937b-118d8f878e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:58.916078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424145c-c822-49dc-9738-d255f21c8ad6 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:58.916397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5f7f5-9ea8-4559-937b-118d8f878e0b service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:58.921606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:58.921903 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:58.925270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:58.925374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:58.925472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:58.925559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:58.925643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] PATH_INFO: `/v3/projects/c2ec7003d5f444298214b5370f94c0d3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:58.925747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] PATH_INFO: `/v3/projects/66bb3acd2f6842db84d86d246f0414f4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:58.925833 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ec7003d5f444298214b5370f94c0d3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:58.926725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bb3acd2f6842db84d86d246f0414f4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:58.927284 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:58.928890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:59.048839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bb3acd2f6842db84d86d246f0414f4', 'request_id': 'req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.051247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ec7003d5f444298214b5370f94c0d3', 'request_id': 'req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.056763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bb3acd2f6842db84d86d246f0414f4', 'request_id': 'req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.059793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ec7003d5f444298214b5370f94c0d3', 'request_id': 'req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.063828 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bb3acd2f6842db84d86d246f0414f4', 'request_id': 'req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.069025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ec7003d5f444298214b5370f94c0d3', 'request_id': 'req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.134467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:59.167370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] The token cache is being invalidate because project 66bb3acd2f6842db84d86d246f0414f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:59.167370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.167370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.167370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1003/1995] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:58 2022] DELETE /identity/v3/projects/66bb3acd2f6842db84d86d246f0414f4 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:56:59.167370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:59.167844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:59.219059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] The token cache is being invalidate because project c2ec7003d5f444298214b5370f94c0d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:59.219637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.220114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.222341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 993/1996] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:58 2022] DELETE /identity/v3/projects/c2ec7003d5f444298214b5370f94c0d3 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:59.227031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:59.229285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:59.233369 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b230f-2faf-4d12-a5b1-8cdc16e6395f admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:59.238849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:59.239722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:59.240130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:59.240464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] PATH_INFO: `/v3/projects/25c8d1a9a0d64933bdcd65ee2edda27a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:59.241731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c8d1a9a0d64933bdcd65ee2edda27a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:59.247952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:59.301292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb192cd-83b8-4fae-8a0e-2ad0dd60cc30 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:59.306209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:59.307109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:59.307545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:59.308480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] PATH_INFO: `/v3/users/7ebf54444a574b5f8fe311c3176d2373` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:59.309903 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ebf54444a574b5f8fe311c3176d2373)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:59.311957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:59.354849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c8d1a9a0d64933bdcd65ee2edda27a', 'request_id': 'req-76e63852-eb44-4452-bd9b-42e4c595cc8d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.363576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c8d1a9a0d64933bdcd65ee2edda27a', 'request_id': 'req-76e63852-eb44-4452-bd9b-42e4c595cc8d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.370035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c8d1a9a0d64933bdcd65ee2edda27a', 'request_id': 'req-76e63852-eb44-4452-bd9b-42e4c595cc8d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.459036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:56:59.468392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] The token cache is being invalidate because project 25c8d1a9a0d64933bdcd65ee2edda27a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:56:59.469158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.469997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-76e63852-eb44-4452-bd9b-42e4c595cc8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.475854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1004/1997] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:56:59 2022] DELETE /identity/v3/projects/25c8d1a9a0d64933bdcd65ee2edda27a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:59.477127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b65c3-37a7-43ec-b6a2-3896aa6dbebf None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:59.477427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b65c3-37a7-43ec-b6a2-3896aa6dbebf None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:59.477711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b65c3-37a7-43ec-b6a2-3896aa6dbebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:59.497941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-244b65c3-37a7-43ec-b6a2-3896aa6dbebf None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:59.499190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-244b65c3-37a7-43ec-b6a2-3896aa6dbebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:59.524006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ebf54444a574b5f8fe311c3176d2373'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.524955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ebf54444a574b5f8fe311c3176d2373'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.542244 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ebf54444a574b5f8fe311c3176d2373'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.543308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ebf54444a574b5f8fe311c3176d2373'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.557975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ebf54444a574b5f8fe311c3176d2373'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:56:59.598216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1005/1998] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:59.604540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0a4db-3c67-43ef-bf59-f7098b8bc684 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:59.604540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0a4db-3c67-43ef-bf59-f7098b8bc684 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:59.604540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0a4db-3c67-43ef-bf59-f7098b8bc684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:59.620116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-f4a0a4db-3c67-43ef-bf59-f7098b8bc684 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:59.621364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f4a0a4db-3c67-43ef-bf59-f7098b8bc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:59.629470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1006/1999] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:59.630763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7efe4-b9d6-4da6-83a4-982d2ef23a0a None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:59.631050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7efe4-b9d6-4da6-83a4-982d2ef23a0a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:59.631408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7efe4-b9d6-4da6-83a4-982d2ef23a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:59.651019 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-5bf7efe4-b9d6-4da6-83a4-982d2ef23a0a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:59.652406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5bf7efe4-b9d6-4da6-83a4-982d2ef23a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:59.661478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1007/2000] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:59.662700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f01be-bb1e-4227-90dd-a07e30da996e None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:59.663114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f01be-bb1e-4227-90dd-a07e30da996e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:59.663450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f01be-bb1e-4227-90dd-a07e30da996e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:59.674110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 994/2001] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:56:59 2022] DELETE /identity/v3/users/7ebf54444a574b5f8fe311c3176d2373 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:56:59.675259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:59.685265 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a35f01be-bb1e-4227-90dd-a07e30da996e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:56:59.686639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a35f01be-bb1e-4227-90dd-a07e30da996e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:56:59.761475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1008/2002] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:56:59.761685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89af27-2183-4ceb-82bd-53cce22046ed admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:59.765875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f01be-bb1e-4227-90dd-a07e30da996e None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:59.774664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfff0e-0c21-4277-af2c-f46d6e62fae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:59.776037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfff0e-0c21-4277-af2c-f46d6e62fae2 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:59.776337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfff0e-0c21-4277-af2c-f46d6e62fae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:59.776691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfff0e-0c21-4277-af2c-f46d6e62fae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:59.777442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfff0e-0c21-4277-af2c-f46d6e62fae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:59.777963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f01be-bb1e-4227-90dd-a07e30da996e None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:59.783968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d34db-dc79-4766-9596-a651a374ce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:59.785615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d34db-dc79-4766-9596-a651a374ce62 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:59.785966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d34db-dc79-4766-9596-a651a374ce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:59.786255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d34db-dc79-4766-9596-a651a374ce62 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:59.787123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d34db-dc79-4766-9596-a651a374ce62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:56:59.789269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d34db-dc79-4766-9596-a651a374ce62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:59.789477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfff0e-0c21-4277-af2c-f46d6e62fae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:59.789559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d34db-dc79-4766-9596-a651a374ce62 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:59.792900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfff0e-0c21-4277-af2c-f46d6e62fae2 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:59.808853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1009/2003] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:59.812733 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d34db-dc79-4766-9596-a651a374ce62 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:59.816755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 995/2004] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:56:59.821543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfff0e-0c21-4277-af2c-f46d6e62fae2 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:59.831867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d34db-dc79-4766-9596-a651a374ce62 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:59.837181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfff0e-0c21-4277-af2c-f46d6e62fae2 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:56:59.845363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e85af-2285-45ad-9910-ee4aeec157bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:59.845686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffffea-df63-4c78-9fb1-32072cac3608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:56:59.848111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e85af-2285-45ad-9910-ee4aeec157bd admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:59.848111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e85af-2285-45ad-9910-ee4aeec157bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:59.848111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e85af-2285-45ad-9910-ee4aeec157bd admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:59.848331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffffea-df63-4c78-9fb1-32072cac3608 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:56:59.848331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffffea-df63-4c78-9fb1-32072cac3608 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:56:59.848505 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e85af-2285-45ad-9910-ee4aeec157bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:59.848577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffffea-df63-4c78-9fb1-32072cac3608 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:56:59.849175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffffea-df63-4c78-9fb1-32072cac3608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:56:59.850219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e85af-2285-45ad-9910-ee4aeec157bd admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:59.851093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffffea-df63-4c78-9fb1-32072cac3608 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:56:59.990555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 996/2006] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:59 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:56:59.993537 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1010/2006] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:56:59 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:56:59.994304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffffea-df63-4c78-9fb1-32072cac3608 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:56:59.994417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e85af-2285-45ad-9910-ee4aeec157bd admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:00.085029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffffea-df63-4c78-9fb1-32072cac3608 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.090601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9f86b-4de6-4910-ad84-c8cbdbaf93c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.091520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9f86b-4de6-4910-ad84-c8cbdbaf93c0 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.092194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9f86b-4de6-4910-ad84-c8cbdbaf93c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.092674 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9f86b-4de6-4910-ad84-c8cbdbaf93c0 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.093386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9f86b-4de6-4910-ad84-c8cbdbaf93c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.095330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9f86b-4de6-4910-ad84-c8cbdbaf93c0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:00.095702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e85af-2285-45ad-9910-ee4aeec157bd admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.101454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34a0cd-c389-4c88-810a-275d23ba9091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.102277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34a0cd-c389-4c88-810a-275d23ba9091 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.102564 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34a0cd-c389-4c88-810a-275d23ba9091 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.102863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34a0cd-c389-4c88-810a-275d23ba9091 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.103740 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34a0cd-c389-4c88-810a-275d23ba9091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.105810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34a0cd-c389-4c88-810a-275d23ba9091 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:00.221469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1011/2007] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:59 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:00.221469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34a0cd-c389-4c88-810a-275d23ba9091 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:00.222255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 997/2008] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:56:59 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:00.223308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9f86b-4de6-4910-ad84-c8cbdbaf93c0 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:00.224231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ca34a0cd-c389-4c88-810a-275d23ba9091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:00.237169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34a0cd-c389-4c88-810a-275d23ba9091 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.237750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9f86b-4de6-4910-ad84-c8cbdbaf93c0 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.242510 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f442af7-d43b-4da2-acb0-4b7fca476260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.243036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.243335 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442af7-d43b-4da2-acb0-4b7fca476260 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.243691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442af7-d43b-4da2-acb0-4b7fca476260 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.244405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442af7-d43b-4da2-acb0-4b7fca476260 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.244537 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.244537 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.244831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] PATH_INFO: `/v3/projects/e5a13bda976b4bd0868f3923c80d3894` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.245046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f442af7-d43b-4da2-acb0-4b7fca476260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.246212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5a13bda976b4bd0868f3923c80d3894)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.246845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f442af7-d43b-4da2-acb0-4b7fca476260 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:00.248248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:00.261998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 998/2009] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:00.263149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f442af7-d43b-4da2-acb0-4b7fca476260 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:00.277247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f442af7-d43b-4da2-acb0-4b7fca476260 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.285434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a77079-6973-4d88-a47b-184091b040d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.286185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a77079-6973-4d88-a47b-184091b040d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.286633 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a77079-6973-4d88-a47b-184091b040d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.286914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a77079-6973-4d88-a47b-184091b040d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.287649 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a77079-6973-4d88-a47b-184091b040d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.289295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a77079-6973-4d88-a47b-184091b040d5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:00.300570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 999/2010] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:00.304062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a77079-6973-4d88-a47b-184091b040d5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:00.313474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a77079-6973-4d88-a47b-184091b040d5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.318122 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2832973-16dc-408e-a8fd-7f9bcb06c279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.318811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2832973-16dc-408e-a8fd-7f9bcb06c279 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.319094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2832973-16dc-408e-a8fd-7f9bcb06c279 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.319370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2832973-16dc-408e-a8fd-7f9bcb06c279 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.320079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2832973-16dc-408e-a8fd-7f9bcb06c279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.321835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2832973-16dc-408e-a8fd-7f9bcb06c279 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:00.331884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1000/2011] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:00.332731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2832973-16dc-408e-a8fd-7f9bcb06c279 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:00.341846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2832973-16dc-408e-a8fd-7f9bcb06c279 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.346307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece2498-322b-41eb-9518-a93c75a08641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.347469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece2498-322b-41eb-9518-a93c75a08641 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.347747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece2498-322b-41eb-9518-a93c75a08641 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.348020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece2498-322b-41eb-9518-a93c75a08641 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.348734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece2498-322b-41eb-9518-a93c75a08641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.350283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece2498-322b-41eb-9518-a93c75a08641 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:00.359694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1001/2012] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:00.363312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece2498-322b-41eb-9518-a93c75a08641 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:00.375701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece2498-322b-41eb-9518-a93c75a08641 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.389254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5a13bda976b4bd0868f3923c80d3894', 'request_id': 'req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:00.397204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5a13bda976b4bd0868f3923c80d3894', 'request_id': 'req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:00.403274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5a13bda976b4bd0868f3923c80d3894', 'request_id': 'req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:00.443974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03645fe-e524-4f5e-b8b3-27634cf622bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.444892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03645fe-e524-4f5e-b8b3-27634cf622bb admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.445176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03645fe-e524-4f5e-b8b3-27634cf622bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.445449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03645fe-e524-4f5e-b8b3-27634cf622bb admin admin] PATH_INFO: `/v3/projects/ef23d64217d24ceea3e499abd13976fd/users/90ce6ce6421c4beb973af5d439e69351/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.462940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03645fe-e524-4f5e-b8b3-27634cf622bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef23d64217d24ceea3e499abd13976fd, user_id=90ce6ce6421c4beb973af5d439e69351, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.467349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03645fe-e524-4f5e-b8b3-27634cf622bb admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:00.479131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:00.483928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] The token cache is being invalidate because project e5a13bda976b4bd0868f3923c80d3894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:00.484302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:00.485059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:00.486697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1012/2013] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:00 2022] DELETE /identity/v3/projects/e5a13bda976b4bd0868f3923c80d3894 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:00.489174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:00.490526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:00.503198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7eb05-f6c6-4b49-bb7f-223d69f45b63 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.507818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abc367-3782-44a6-8166-a8f9f352c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.508725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abc367-3782-44a6-8166-a8f9f352c643 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.509016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abc367-3782-44a6-8166-a8f9f352c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.509297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abc367-3782-44a6-8166-a8f9f352c643 admin admin] PATH_INFO: `/v3/projects/11f303059bc34edfba1b471d8e862e23/users/59127a5d464949eaa422fd9166db89db/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.524387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abc367-3782-44a6-8166-a8f9f352c643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f303059bc34edfba1b471d8e862e23, user_id=59127a5d464949eaa422fd9166db89db, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.527047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abc367-3782-44a6-8166-a8f9f352c643 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:00.569359 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1002/2014] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:00 2022] PUT /identity/v3/projects/ef23d64217d24ceea3e499abd13976fd/users/90ce6ce6421c4beb973af5d439e69351/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:00.579785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03645fe-e524-4f5e-b8b3-27634cf622bb admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:00.579785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a03645fe-e524-4f5e-b8b3-27634cf622bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:00.612048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1013/2015] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:00 2022] PUT /identity/v3/projects/11f303059bc34edfba1b471d8e862e23/users/59127a5d464949eaa422fd9166db89db/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:00.612048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f0930-ac5d-4c0a-a50b-e037366f66b6 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.612048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f0930-ac5d-4c0a-a50b-e037366f66b6 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.612048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f0930-ac5d-4c0a-a50b-e037366f66b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.638934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-5c2f0930-ac5d-4c0a-a50b-e037366f66b6 None None] MFA Rules not processed for user `90ce6ce6421c4beb973af5d439e69351`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:00.640767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5c2f0930-ac5d-4c0a-a50b-e037366f66b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:00.654483 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03645fe-e524-4f5e-b8b3-27634cf622bb admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.735738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:00.744536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.744536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.744536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.744536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] PATH_INFO: `/v3/users/2d9ca85f7ba64069823ee9667ffe9f02` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.745757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d9ca85f7ba64069823ee9667ffe9f02)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.747588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:00.757811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1014/2016] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:00.760060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590758-efcb-4316-b2c5-7a8b162955cd None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.760377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590758-efcb-4316-b2c5-7a8b162955cd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.760657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590758-efcb-4316-b2c5-7a8b162955cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.806022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d0590758-efcb-4316-b2c5-7a8b162955cd None None] MFA Rules not processed for user `59127a5d464949eaa422fd9166db89db`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:00.807306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d0590758-efcb-4316-b2c5-7a8b162955cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:00.880488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1015/2017] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:00.881395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0590758-efcb-4316-b2c5-7a8b162955cd None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:00.882680 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d0590758-efcb-4316-b2c5-7a8b162955cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:00.896331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0590758-efcb-4316-b2c5-7a8b162955cd None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:00.901330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbb4cc-7bbc-4a69-be1a-3d456960fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:00.902009 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb4cc-7bbc-4a69-be1a-3d456960fc64 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:00.902308 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb4cc-7bbc-4a69-be1a-3d456960fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:00.902577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbb4cc-7bbc-4a69-be1a-3d456960fc64 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:00.903263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbb4cc-7bbc-4a69-be1a-3d456960fc64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:00.905214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbb4cc-7bbc-4a69-be1a-3d456960fc64 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.006510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1016/2018] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:00 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:01.006510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbb4cc-7bbc-4a69-be1a-3d456960fc64 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.065078 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbb4cc-7bbc-4a69-be1a-3d456960fc64 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.070466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f04a00-7195-4110-b609-ca10846f2bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.071139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f04a00-7195-4110-b609-ca10846f2bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.071422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f04a00-7195-4110-b609-ca10846f2bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.071703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f04a00-7195-4110-b609-ca10846f2bf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.072458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f04a00-7195-4110-b609-ca10846f2bf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.074364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f04a00-7195-4110-b609-ca10846f2bf7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.095714 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d9ca85f7ba64069823ee9667ffe9f02'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:01.096385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d9ca85f7ba64069823ee9667ffe9f02'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:01.107884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d9ca85f7ba64069823ee9667ffe9f02'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:01.108543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d9ca85f7ba64069823ee9667ffe9f02'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:01.116013 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d9ca85f7ba64069823ee9667ffe9f02'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:01.247683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1003/2019] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:00 2022] DELETE /identity/v3/users/2d9ca85f7ba64069823ee9667ffe9f02 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:01.249448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.251557 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1017/2020] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:01 2022] POST /identity/v3/projects => generated 375 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:01.252530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f04a00-7195-4110-b609-ca10846f2bf7 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.254740 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-56f04a00-7195-4110-b609-ca10846f2bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:01.257748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:01.328851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788afc2-40e4-40b3-afee-d226acf71eef admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.335310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece4f6d-7546-4c7d-b0f3-5a5ecc46eb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.338191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece4f6d-7546-4c7d-b0f3-5a5ecc46eb30 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.338687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece4f6d-7546-4c7d-b0f3-5a5ecc46eb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.339003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece4f6d-7546-4c7d-b0f3-5a5ecc46eb30 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.339735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece4f6d-7546-4c7d-b0f3-5a5ecc46eb30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.340540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f04a00-7195-4110-b609-ca10846f2bf7 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.341767 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece4f6d-7546-4c7d-b0f3-5a5ecc46eb30 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.346145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fea8c-5e61-4758-bb31-0ffe9459b817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.346920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fea8c-5e61-4758-bb31-0ffe9459b817 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.351425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fea8c-5e61-4758-bb31-0ffe9459b817 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.351425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fea8c-5e61-4758-bb31-0ffe9459b817 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.351425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fea8c-5e61-4758-bb31-0ffe9459b817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.351425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fea8c-5e61-4758-bb31-0ffe9459b817 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.420045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1018/2021] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:01 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:01.420346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1004/2022] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:01 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:01.423827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece4f6d-7546-4c7d-b0f3-5a5ecc46eb30 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.427717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fea8c-5e61-4758-bb31-0ffe9459b817 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.437353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece4f6d-7546-4c7d-b0f3-5a5ecc46eb30 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.448603 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346dd731-8dba-4ae0-af3b-ca852164f9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.449693 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346dd731-8dba-4ae0-af3b-ca852164f9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.450199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346dd731-8dba-4ae0-af3b-ca852164f9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.450306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fea8c-5e61-4758-bb31-0ffe9459b817 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.450742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346dd731-8dba-4ae0-af3b-ca852164f9f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.454228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346dd731-8dba-4ae0-af3b-ca852164f9f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.454598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346dd731-8dba-4ae0-af3b-ca852164f9f6 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.455685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b890cf-2131-4abb-8a81-519d26177882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.456385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b890cf-2131-4abb-8a81-519d26177882 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.456825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b890cf-2131-4abb-8a81-519d26177882 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.457000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b890cf-2131-4abb-8a81-519d26177882 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.457758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b890cf-2131-4abb-8a81-519d26177882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.459445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b890cf-2131-4abb-8a81-519d26177882 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.469797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1005/2023] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:01.475215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346dd731-8dba-4ae0-af3b-ca852164f9f6 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.478153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1019/2024] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:01.489272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346dd731-8dba-4ae0-af3b-ca852164f9f6 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.491794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b890cf-2131-4abb-8a81-519d26177882 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.494281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21dabd-1457-4c36-a300-3fe3556c1430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.495261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21dabd-1457-4c36-a300-3fe3556c1430 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.495619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21dabd-1457-4c36-a300-3fe3556c1430 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.495970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21dabd-1457-4c36-a300-3fe3556c1430 admin admin] PATH_INFO: `/v3/projects/0010d15f7c554420b8f76c04d868becc/users/daba0bb0ec3d4576be063180430c8bef/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.502310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b890cf-2131-4abb-8a81-519d26177882 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.508517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4debb-fb26-491f-a9d5-9ede372d050d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.509206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4debb-fb26-491f-a9d5-9ede372d050d admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.509846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4debb-fb26-491f-a9d5-9ede372d050d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.510343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4debb-fb26-491f-a9d5-9ede372d050d admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.511166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4debb-fb26-491f-a9d5-9ede372d050d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.513766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4debb-fb26-491f-a9d5-9ede372d050d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.521894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21dabd-1457-4c36-a300-3fe3556c1430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0010d15f7c554420b8f76c04d868becc, user_id=daba0bb0ec3d4576be063180430c8bef, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.521894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21dabd-1457-4c36-a300-3fe3556c1430 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.528702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1020/2025] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:01.532960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4debb-fb26-491f-a9d5-9ede372d050d admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.545711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4debb-fb26-491f-a9d5-9ede372d050d admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.551459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59564620-1908-4955-818d-88bef9339d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.552416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59564620-1908-4955-818d-88bef9339d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.552700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59564620-1908-4955-818d-88bef9339d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.552978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59564620-1908-4955-818d-88bef9339d15 admin admin] PATH_INFO: `/v3/projects/0211feb02bc14c428eace423c75dd094/users/5faf2fb6460e467b803d6ba452fb03ed/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.569142 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59564620-1908-4955-818d-88bef9339d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0211feb02bc14c428eace423c75dd094, user_id=5faf2fb6460e467b803d6ba452fb03ed, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.571743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59564620-1908-4955-818d-88bef9339d15 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.590759 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1006/2026] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:01 2022] PUT /identity/v3/projects/0010d15f7c554420b8f76c04d868becc/users/daba0bb0ec3d4576be063180430c8bef/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:01.594469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21dabd-1457-4c36-a300-3fe3556c1430 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.660902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21dabd-1457-4c36-a300-3fe3556c1430 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.665503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b9d70-cff3-4d9a-aaf9-0dfc1578096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.666169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b9d70-cff3-4d9a-aaf9-0dfc1578096d admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.666448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b9d70-cff3-4d9a-aaf9-0dfc1578096d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.666726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b9d70-cff3-4d9a-aaf9-0dfc1578096d admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.667449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b9d70-cff3-4d9a-aaf9-0dfc1578096d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.669148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b9d70-cff3-4d9a-aaf9-0dfc1578096d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.676061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1021/2027] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:01 2022] PUT /identity/v3/projects/0211feb02bc14c428eace423c75dd094/users/5faf2fb6460e467b803d6ba452fb03ed/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:01.679495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1007/2028] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:01.683573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59564620-1908-4955-818d-88bef9339d15 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.686052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9382825-786a-4033-9634-558050681878 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.690310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9382825-786a-4033-9634-558050681878 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.690310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9382825-786a-4033-9634-558050681878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.713305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a9382825-786a-4033-9634-558050681878 None None] MFA Rules not processed for user `5faf2fb6460e467b803d6ba452fb03ed`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:01.714507 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a9382825-786a-4033-9634-558050681878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:01.724731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:01.762745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59564620-1908-4955-818d-88bef9339d15 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.767529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953ae94-5af5-4bc3-a7c5-652c67abcad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.768459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953ae94-5af5-4bc3-a7c5-652c67abcad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.768748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953ae94-5af5-4bc3-a7c5-652c67abcad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.769031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953ae94-5af5-4bc3-a7c5-652c67abcad1 admin admin] PATH_INFO: `/v3/projects/0010d15f7c554420b8f76c04d868becc/users/daba0bb0ec3d4576be063180430c8bef/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.771681 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953ae94-5af5-4bc3-a7c5-652c67abcad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0010d15f7c554420b8f76c04d868becc, user_id=daba0bb0ec3d4576be063180430c8bef, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.774622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953ae94-5af5-4bc3-a7c5-652c67abcad1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.785718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1022/2029] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:01 2022] PUT /identity/v3/projects/0010d15f7c554420b8f76c04d868becc/users/daba0bb0ec3d4576be063180430c8bef/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:01.786639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953ae94-5af5-4bc3-a7c5-652c67abcad1 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.788183 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6953ae94-5af5-4bc3-a7c5-652c67abcad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:01.801347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1008/2030] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:01.805209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e33abb-df3c-4e07-8db9-1f955447268c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.805209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e33abb-df3c-4e07-8db9-1f955447268c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.805209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e33abb-df3c-4e07-8db9-1f955447268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.836279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-55e33abb-df3c-4e07-8db9-1f955447268c None None] MFA Rules not processed for user `daba0bb0ec3d4576be063180430c8bef`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:01.837587 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-55e33abb-df3c-4e07-8db9-1f955447268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:01.877279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953ae94-5af5-4bc3-a7c5-652c67abcad1 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.884849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.885952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.886571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.886986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] PATH_INFO: `/v3/projects/6391ff96f67241529957a14766c72a91` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.888447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6391ff96f67241529957a14766c72a91)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.890794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:01.933023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1009/2031] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:01.934222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e33abb-df3c-4e07-8db9-1f955447268c None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:01.945402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e33abb-df3c-4e07-8db9-1f955447268c None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:01.948566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:01.952380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbb115-8731-45af-a1ac-2b509d20ded5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:01.953105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbb115-8731-45af-a1ac-2b509d20ded5 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:01.953387 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbb115-8731-45af-a1ac-2b509d20ded5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:01.953725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbb115-8731-45af-a1ac-2b509d20ded5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:01.954535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbb115-8731-45af-a1ac-2b509d20ded5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:01.957441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbb115-8731-45af-a1ac-2b509d20ded5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.022922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6391ff96f67241529957a14766c72a91', 'request_id': 'req-e7e65756-b361-4aaa-b302-2a1f41d435e6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:02.029798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6391ff96f67241529957a14766c72a91', 'request_id': 'req-e7e65756-b361-4aaa-b302-2a1f41d435e6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:02.035507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6391ff96f67241529957a14766c72a91', 'request_id': 'req-e7e65756-b361-4aaa-b302-2a1f41d435e6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:02.086847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1010/2032] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:01 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:02.086847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbb115-8731-45af-a1ac-2b509d20ded5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.091000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-85dbb115-8731-45af-a1ac-2b509d20ded5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:02.126563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] The token cache is being invalidate because project 6391ff96f67241529957a14766c72a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:02.130293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:02.130293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:02.130759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1023/2033] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:01 2022] DELETE /identity/v3/projects/6391ff96f67241529957a14766c72a91 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:02.132090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.133618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:02.166056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbb115-8731-45af-a1ac-2b509d20ded5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.170044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ebb5a-6cea-46d8-9358-c92d893fd46c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.170763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ebb5a-6cea-46d8-9358-c92d893fd46c service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.171044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ebb5a-6cea-46d8-9358-c92d893fd46c service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.171322 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ebb5a-6cea-46d8-9358-c92d893fd46c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.173601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8a4ebb5a-6cea-46d8-9358-c92d893fd46c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:02.250171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ebb5a-6cea-46d8-9358-c92d893fd46c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.251919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ebb5a-6cea-46d8-9358-c92d893fd46c service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.255230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e65756-b361-4aaa-b302-2a1f41d435e6 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.262035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1011/2034] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:02.263419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3951b0-a677-4f07-90b1-ba3f8b100ae7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.263711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3951b0-a677-4f07-90b1-ba3f8b100ae7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.263992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3951b0-a677-4f07-90b1-ba3f8b100ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.287757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1f3951b0-a677-4f07-90b1-ba3f8b100ae7 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:02.291019 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1f3951b0-a677-4f07-90b1-ba3f8b100ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:02.326243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b35ff-2824-4496-bbb7-71a923b13b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.326930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b35ff-2824-4496-bbb7-71a923b13b83 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.327217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b35ff-2824-4496-bbb7-71a923b13b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.327495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b35ff-2824-4496-bbb7-71a923b13b83 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.328227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b35ff-2824-4496-bbb7-71a923b13b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.330237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b35ff-2824-4496-bbb7-71a923b13b83 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.375824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1012/2035] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:02.385394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f3670-446f-46a8-b4ca-88651ddfc2c1 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.385686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f3670-446f-46a8-b4ca-88651ddfc2c1 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.385968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f3670-446f-46a8-b4ca-88651ddfc2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.406132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1f2f3670-446f-46a8-b4ca-88651ddfc2c1 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:02.407406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1f2f3670-446f-46a8-b4ca-88651ddfc2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:02.416028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1013/2036] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:02.420352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f3670-446f-46a8-b4ca-88651ddfc2c1 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.426851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1024/2037] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:02 2022] POST /identity/v3/users => generated 299 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:02.431976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f3670-446f-46a8-b4ca-88651ddfc2c1 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.434051 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b35ff-2824-4496-bbb7-71a923b13b83 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.436788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6761b63-8f15-4329-821e-55e8d8ae74e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.437451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6761b63-8f15-4329-821e-55e8d8ae74e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.437741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6761b63-8f15-4329-821e-55e8d8ae74e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.438025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6761b63-8f15-4329-821e-55e8d8ae74e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.438751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6761b63-8f15-4329-821e-55e8d8ae74e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:02.439058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6761b63-8f15-4329-821e-55e8d8ae74e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.440655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6761b63-8f15-4329-821e-55e8d8ae74e5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.445339 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b35ff-2824-4496-bbb7-71a923b13b83 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.454191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e729f48-1d5b-42c7-a48a-42e5b1550d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.454191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e729f48-1d5b-42c7-a48a-42e5b1550d41 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.454191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e729f48-1d5b-42c7-a48a-42e5b1550d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.454191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e729f48-1d5b-42c7-a48a-42e5b1550d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.454512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e729f48-1d5b-42c7-a48a-42e5b1550d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.455814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1014/2038] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:57:02.456516 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e729f48-1d5b-42c7-a48a-42e5b1550d41 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.460728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6761b63-8f15-4329-821e-55e8d8ae74e5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.470940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1025/2039] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:02.473225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6761b63-8f15-4329-821e-55e8d8ae74e5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.475396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e729f48-1d5b-42c7-a48a-42e5b1550d41 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.478177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7386750-7000-4b6a-8fc9-69bf2c645f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.478846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7386750-7000-4b6a-8fc9-69bf2c645f6c admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.479120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7386750-7000-4b6a-8fc9-69bf2c645f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.479394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7386750-7000-4b6a-8fc9-69bf2c645f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.480086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7386750-7000-4b6a-8fc9-69bf2c645f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.482120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7386750-7000-4b6a-8fc9-69bf2c645f6c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.486821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e729f48-1d5b-42c7-a48a-42e5b1550d41 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.495679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b2a02-8dbe-408b-8403-e8ea14f1fb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.496830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b2a02-8dbe-408b-8403-e8ea14f1fb01 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.497165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b2a02-8dbe-408b-8403-e8ea14f1fb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.497471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b2a02-8dbe-408b-8403-e8ea14f1fb01 admin admin] PATH_INFO: `/v3/projects/20ee46ff0b834238b419fe98922a5d70/users/6cdd90b462394cf193cd4bb4886f2c50/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.512587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b2a02-8dbe-408b-8403-e8ea14f1fb01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20ee46ff0b834238b419fe98922a5d70, user_id=6cdd90b462394cf193cd4bb4886f2c50, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.515270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b2a02-8dbe-408b-8403-e8ea14f1fb01 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.603252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1015/2040] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:02 2022] POST /identity/v3/projects => generated 389 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:02.607486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7386750-7000-4b6a-8fc9-69bf2c645f6c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.609710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1026/2041] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:02 2022] PUT /identity/v3/projects/20ee46ff0b834238b419fe98922a5d70/users/6cdd90b462394cf193cd4bb4886f2c50/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:02.613734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b2a02-8dbe-408b-8403-e8ea14f1fb01 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.680345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7386750-7000-4b6a-8fc9-69bf2c645f6c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.680754 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b2a02-8dbe-408b-8403-e8ea14f1fb01 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.685848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d49277-f41f-4754-9cab-0b586927bf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.686552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d49277-f41f-4754-9cab-0b586927bf61 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.686841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d49277-f41f-4754-9cab-0b586927bf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.686946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392daaf0-6023-4730-9b85-36afb7b3333a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.687114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d49277-f41f-4754-9cab-0b586927bf61 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.687727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392daaf0-6023-4730-9b85-36afb7b3333a admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.687871 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d49277-f41f-4754-9cab-0b586927bf61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.688112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392daaf0-6023-4730-9b85-36afb7b3333a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.688497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392daaf0-6023-4730-9b85-36afb7b3333a admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.689255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392daaf0-6023-4730-9b85-36afb7b3333a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.689484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d49277-f41f-4754-9cab-0b586927bf61 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.694472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392daaf0-6023-4730-9b85-36afb7b3333a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.707875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1027/2042] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:02.713319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d49277-f41f-4754-9cab-0b586927bf61 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.729920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d49277-f41f-4754-9cab-0b586927bf61 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.735157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe9092-7c4a-43b3-9326-49337b6a8af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.736182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe9092-7c4a-43b3-9326-49337b6a8af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.736602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe9092-7c4a-43b3-9326-49337b6a8af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.736943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe9092-7c4a-43b3-9326-49337b6a8af2 admin admin] PATH_INFO: `/v3/projects/20ee46ff0b834238b419fe98922a5d70/users/6cdd90b462394cf193cd4bb4886f2c50/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.739079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe9092-7c4a-43b3-9326-49337b6a8af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20ee46ff0b834238b419fe98922a5d70, user_id=6cdd90b462394cf193cd4bb4886f2c50, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.741853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe9092-7c4a-43b3-9326-49337b6a8af2 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.752539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1028/2043] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:02 2022] PUT /identity/v3/projects/20ee46ff0b834238b419fe98922a5d70/users/6cdd90b462394cf193cd4bb4886f2c50/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:02.763470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edf111-4e34-4cdd-bb19-20daeb4a6ddf None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.766122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:02.766122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edf111-4e34-4cdd-bb19-20daeb4a6ddf None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.766122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edf111-4e34-4cdd-bb19-20daeb4a6ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.795201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1016/2044] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:02 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:02.799085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392daaf0-6023-4730-9b85-36afb7b3333a admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.803251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-36edf111-4e34-4cdd-bb19-20daeb4a6ddf None None] MFA Rules not processed for user `6cdd90b462394cf193cd4bb4886f2c50`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:02.804701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-36edf111-4e34-4cdd-bb19-20daeb4a6ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:02.810095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392daaf0-6023-4730-9b85-36afb7b3333a admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.877475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a352752-b2b6-4f72-978f-7210efa3e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.878295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a352752-b2b6-4f72-978f-7210efa3e09c admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.879515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a352752-b2b6-4f72-978f-7210efa3e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.880024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a352752-b2b6-4f72-978f-7210efa3e09c admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.880950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a352752-b2b6-4f72-978f-7210efa3e09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.882692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a352752-b2b6-4f72-978f-7210efa3e09c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.884685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1029/2045] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:02.892801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36edf111-4e34-4cdd-bb19-20daeb4a6ddf None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.895825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-36edf111-4e34-4cdd-bb19-20daeb4a6ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:02.906860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1017/2046] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:02.906860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a352752-b2b6-4f72-978f-7210efa3e09c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.920552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a352752-b2b6-4f72-978f-7210efa3e09c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.931560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51532768-8569-4d07-ba17-38109822ed9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.932832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51532768-8569-4d07-ba17-38109822ed9d admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.933301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51532768-8569-4d07-ba17-38109822ed9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.934047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51532768-8569-4d07-ba17-38109822ed9d admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.936084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51532768-8569-4d07-ba17-38109822ed9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:02.938718 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51532768-8569-4d07-ba17-38109822ed9d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:02.959546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1018/2047] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:02.961091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51532768-8569-4d07-ba17-38109822ed9d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:02.971478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51532768-8569-4d07-ba17-38109822ed9d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.976310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36edf111-4e34-4cdd-bb19-20daeb4a6ddf None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:02.976796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f38853-1dd8-4665-80ad-ffc0adfd7c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.977762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f38853-1dd8-4665-80ad-ffc0adfd7c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.978115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f38853-1dd8-4665-80ad-ffc0adfd7c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.978463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f38853-1dd8-4665-80ad-ffc0adfd7c5e admin admin] PATH_INFO: `/v3/projects/95d05bb0f3e64ca9bf7a2809e61fdd8e/users/c231a51d2425450381f78671a2d3e0cf/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.980345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4a845-b5d5-47de-bc00-32a7946d78ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:02.981021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4a845-b5d5-47de-bc00-32a7946d78ce service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:02.981307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4a845-b5d5-47de-bc00-32a7946d78ce service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:02.981587 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4a845-b5d5-47de-bc00-32a7946d78ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:02.983473 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-04f4a845-b5d5-47de-bc00-32a7946d78ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:02.985981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:03.000839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f38853-1dd8-4665-80ad-ffc0adfd7c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d05bb0f3e64ca9bf7a2809e61fdd8e, user_id=c231a51d2425450381f78671a2d3e0cf, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:03.005577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f38853-1dd8-4665-80ad-ffc0adfd7c5e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:03.050703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4a845-b5d5-47de-bc00-32a7946d78ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:03.052813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4a845-b5d5-47de-bc00-32a7946d78ce service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:03.075671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1019/2048] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:02 2022] PUT /identity/v3/projects/95d05bb0f3e64ca9bf7a2809e61fdd8e/users/c231a51d2425450381f78671a2d3e0cf/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:03.078762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1030/2049] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:57:03.089553 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca80c9-24ca-4b29-bf7f-f447c90d3f5c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:03.091755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca80c9-24ca-4b29-bf7f-f447c90d3f5c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:03.092117 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca80c9-24ca-4b29-bf7f-f447c90d3f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:03.134969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9bca80c9-24ca-4b29-bf7f-f447c90d3f5c None None] MFA Rules not processed for user `c231a51d2425450381f78671a2d3e0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:03.136476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9bca80c9-24ca-4b29-bf7f-f447c90d3f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:03.250865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1020/2050] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:03.255693 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4a845-b5d5-47de-bc00-32a7946d78ce service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:03.267438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4a845-b5d5-47de-bc00-32a7946d78ce service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:03.334159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623e959-a6db-49a3-b78d-ca6e68f741cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:03.335307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623e959-a6db-49a3-b78d-ca6e68f741cd admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:03.335593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623e959-a6db-49a3-b78d-ca6e68f741cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:03.336156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623e959-a6db-49a3-b78d-ca6e68f741cd admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:03.337248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623e959-a6db-49a3-b78d-ca6e68f741cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:03.338846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623e959-a6db-49a3-b78d-ca6e68f741cd admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:03.471968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1031/2051] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:03 2022] POST /identity/v3/projects => generated 389 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:03.473737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca80c9-24ca-4b29-bf7f-f447c90d3f5c None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:03.485619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca80c9-24ca-4b29-bf7f-f447c90d3f5c None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:03.553372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394b37d-877a-45ca-980b-930d3a8d769c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:03.554222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394b37d-877a-45ca-980b-930d3a8d769c admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:03.554427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394b37d-877a-45ca-980b-930d3a8d769c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:03.554781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394b37d-877a-45ca-980b-930d3a8d769c admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:03.555407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394b37d-877a-45ca-980b-930d3a8d769c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:03.557362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394b37d-877a-45ca-980b-930d3a8d769c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:03.671206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1021/2052] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:03 2022] POST /identity/v3/users => generated 301 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:03.678216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623e959-a6db-49a3-b78d-ca6e68f741cd admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:03.688663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623e959-a6db-49a3-b78d-ca6e68f741cd admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:03.697091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad03d0a-f452-45ae-80d2-a4f54142becf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:03.697949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad03d0a-f452-45ae-80d2-a4f54142becf admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:03.699936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad03d0a-f452-45ae-80d2-a4f54142becf admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:03.699936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad03d0a-f452-45ae-80d2-a4f54142becf admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:03.699936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad03d0a-f452-45ae-80d2-a4f54142becf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:03.701658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad03d0a-f452-45ae-80d2-a4f54142becf admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:03.717545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1032/2053] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:03.717545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:03.718767 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394b37d-877a-45ca-980b-930d3a8d769c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:03.730233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394b37d-877a-45ca-980b-930d3a8d769c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:03.735012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0b5e4-6aaa-4b40-af4f-7881917b21f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:03.740676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0b5e4-6aaa-4b40-af4f-7881917b21f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:03.740676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0b5e4-6aaa-4b40-af4f-7881917b21f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:03.740676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0b5e4-6aaa-4b40-af4f-7881917b21f7 admin admin] PATH_INFO: `/v3/projects/cd01c277390f4c808307cc71b28718d2/users/39d426716c2e446da702c718fab1d243/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:03.753045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0b5e4-6aaa-4b40-af4f-7881917b21f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd01c277390f4c808307cc71b28718d2, user_id=39d426716c2e446da702c718fab1d243, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:03.756268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0b5e4-6aaa-4b40-af4f-7881917b21f7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:03.974816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1022/2054] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:03 2022] PUT /identity/v3/projects/cd01c277390f4c808307cc71b28718d2/users/39d426716c2e446da702c718fab1d243/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:03.979049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad03d0a-f452-45ae-80d2-a4f54142becf admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:03.989654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad03d0a-f452-45ae-80d2-a4f54142becf admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:03.997763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:04.202216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd33b9-3159-4adb-a006-0f32d1ec7d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:04.202216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd33b9-3159-4adb-a006-0f32d1ec7d40 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:04.202216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd33b9-3159-4adb-a006-0f32d1ec7d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:04.202216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd33b9-3159-4adb-a006-0f32d1ec7d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:04.202894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd33b9-3159-4adb-a006-0f32d1ec7d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:04.202894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd33b9-3159-4adb-a006-0f32d1ec7d40 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:04.202894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1033/2055] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:04.203300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0b5e4-6aaa-4b40-af4f-7881917b21f7 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:04.209097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0b5e4-6aaa-4b40-af4f-7881917b21f7 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:04.213609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2a181-3010-4b4f-8cd8-62b0d09d9d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:04.214484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2a181-3010-4b4f-8cd8-62b0d09d9d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:04.214768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2a181-3010-4b4f-8cd8-62b0d09d9d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:04.215044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2a181-3010-4b4f-8cd8-62b0d09d9d04 admin admin] PATH_INFO: `/v3/projects/cd01c277390f4c808307cc71b28718d2/users/39d426716c2e446da702c718fab1d243/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:04.217011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2a181-3010-4b4f-8cd8-62b0d09d9d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd01c277390f4c808307cc71b28718d2, user_id=39d426716c2e446da702c718fab1d243, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:04.220028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2a181-3010-4b4f-8cd8-62b0d09d9d04 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:04.230355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1023/2056] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:04 2022] PUT /identity/v3/projects/cd01c277390f4c808307cc71b28718d2/users/39d426716c2e446da702c718fab1d243/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:04.243868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c262cd-06a4-4eac-a743-8ae2695834a0 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:04.244167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c262cd-06a4-4eac-a743-8ae2695834a0 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:04.244488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c262cd-06a4-4eac-a743-8ae2695834a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:04.272609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-86c262cd-06a4-4eac-a743-8ae2695834a0 None None] MFA Rules not processed for user `39d426716c2e446da702c718fab1d243`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:04.274134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-86c262cd-06a4-4eac-a743-8ae2695834a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:04.359517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1034/2057] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:04.566785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2a181-3010-4b4f-8cd8-62b0d09d9d04 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:04.627396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2a181-3010-4b4f-8cd8-62b0d09d9d04 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:04.631302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354abeb-8547-4861-a996-99818eaab132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:04.634186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354abeb-8547-4861-a996-99818eaab132 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:04.634186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354abeb-8547-4861-a996-99818eaab132 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:04.634186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354abeb-8547-4861-a996-99818eaab132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:04.642287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354abeb-8547-4861-a996-99818eaab132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:04.643727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354abeb-8547-4861-a996-99818eaab132 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:04.654918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1024/2058] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:04.837978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c262cd-06a4-4eac-a743-8ae2695834a0 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:04.903066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c262cd-06a4-4eac-a743-8ae2695834a0 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:04.907648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:04.908698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:04.908988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:04.909261 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] PATH_INFO: `/v3/users/c231a51d2425450381f78671a2d3e0cf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:04.911119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c231a51d2425450381f78671a2d3e0cf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:04.913189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:05.027433 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:05.119675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c231a51d2425450381f78671a2d3e0cf'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:05.120334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c231a51d2425450381f78671a2d3e0cf'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:05.121112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c231a51d2425450381f78671a2d3e0cf'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:05.130024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c231a51d2425450381f78671a2d3e0cf'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:05.144313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c231a51d2425450381f78671a2d3e0cf'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:05.180670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:05.243874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1035/2059] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:04 2022] DELETE /identity/v3/users/c231a51d2425450381f78671a2d3e0cf => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:05.247016 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb09b9e-5f98-4754-ad75-5a29bf6ede88 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:05.247314 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb09b9e-5f98-4754-ad75-5a29bf6ede88 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:05.247594 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb09b9e-5f98-4754-ad75-5a29bf6ede88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:05.266792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-0eb09b9e-5f98-4754-ad75-5a29bf6ede88 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:05.272987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0eb09b9e-5f98-4754-ad75-5a29bf6ede88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:05.337915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1025/2060] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:05.348149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:05.352665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:05.414986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197ccb0-2f93-4e46-b167-4fabf0cb4c7e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:05.419384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dfc69-b2ad-434b-aaba-fdfd32b8a6b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:05.420047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dfc69-b2ad-434b-aaba-fdfd32b8a6b6 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:05.420332 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dfc69-b2ad-434b-aaba-fdfd32b8a6b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:05.420644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dfc69-b2ad-434b-aaba-fdfd32b8a6b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:05.433638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dfc69-b2ad-434b-aaba-fdfd32b8a6b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:05.435110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dfc69-b2ad-434b-aaba-fdfd32b8a6b6 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:05.445951 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1036/2061] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:05.706325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:05.820074 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb09b9e-5f98-4754-ad75-5a29bf6ede88 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:05.821541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0eb09b9e-5f98-4754-ad75-5a29bf6ede88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:05.835048 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb09b9e-5f98-4754-ad75-5a29bf6ede88 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:05.850675 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:05.851575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:05.852036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:05.852533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] PATH_INFO: `/v3/users/59127a5d464949eaa422fd9166db89db` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:05.855918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59127a5d464949eaa422fd9166db89db)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:05.856280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:05.964758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dfc69-b2ad-434b-aaba-fdfd32b8a6b6 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:05.966196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bc8dfc69-b2ad-434b-aaba-fdfd32b8a6b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:05.979883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dfc69-b2ad-434b-aaba-fdfd32b8a6b6 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:05.987748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:05.987748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:05.987748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:05.987748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] PATH_INFO: `/v3/projects/95d05bb0f3e64ca9bf7a2809e61fdd8e` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:05.988993 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d05bb0f3e64ca9bf7a2809e61fdd8e)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:05.989842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:06.076772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:06.091319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59127a5d464949eaa422fd9166db89db'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.092015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59127a5d464949eaa422fd9166db89db'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.107247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59127a5d464949eaa422fd9166db89db'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.107953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59127a5d464949eaa422fd9166db89db'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.116684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0e960cca-38a0-4152-a9e0-0c335f976d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59127a5d464949eaa422fd9166db89db'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.124688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d05bb0f3e64ca9bf7a2809e61fdd8e', 'request_id': 'req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.130975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d05bb0f3e64ca9bf7a2809e61fdd8e', 'request_id': 'req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.136903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d05bb0f3e64ca9bf7a2809e61fdd8e', 'request_id': 'req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.188142 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1026/2062] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:05 2022] DELETE /identity/v3/users/59127a5d464949eaa422fd9166db89db => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:06.192776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0eedca-4258-4609-ab0b-54947c9cd1cb None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:06.193075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0eedca-4258-4609-ab0b-54947c9cd1cb None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:06.193807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0eedca-4258-4609-ab0b-54947c9cd1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:06.214389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-aa0eedca-4258-4609-ab0b-54947c9cd1cb None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:06.215747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-aa0eedca-4258-4609-ab0b-54947c9cd1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:06.250188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:06.280952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1027/2063] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:06.291048 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0eedca-4258-4609-ab0b-54947c9cd1cb None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:06.292472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-aa0eedca-4258-4609-ab0b-54947c9cd1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:06.307313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] The token cache is being invalidate because project 95d05bb0f3e64ca9bf7a2809e61fdd8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:06.307618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.311534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.313310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1037/2064] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:05 2022] DELETE /identity/v3/projects/95d05bb0f3e64ca9bf7a2809e61fdd8e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:06.317514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:06.319724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:06.357881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0eedca-4258-4609-ab0b-54947c9cd1cb None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:06.373996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac61381-6d7b-4a82-821a-4ee415132138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:06.386602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac61381-6d7b-4a82-821a-4ee415132138 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:06.386602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac61381-6d7b-4a82-821a-4ee415132138 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:06.386602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac61381-6d7b-4a82-821a-4ee415132138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:06.386602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-aac61381-6d7b-4a82-821a-4ee415132138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:06.390329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac61381-6d7b-4a82-821a-4ee415132138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:06.392190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac61381-6d7b-4a82-821a-4ee415132138 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:06.399508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ff1c5-31ba-4b68-9407-b89425d14ac3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:06.404268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1028/2065] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:57:06.419812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:06.420695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:06.420984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:06.421262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] PATH_INFO: `/v3/users/39d426716c2e446da702c718fab1d243` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:06.423331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d426716c2e446da702c718fab1d243)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:06.431579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:06.483749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac61381-6d7b-4a82-821a-4ee415132138 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:06.485814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-aac61381-6d7b-4a82-821a-4ee415132138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:06.555334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac61381-6d7b-4a82-821a-4ee415132138 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:06.559310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d459a-1de7-4ce1-8281-edfde640ea43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:06.560287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d459a-1de7-4ce1-8281-edfde640ea43 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:06.560669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d459a-1de7-4ce1-8281-edfde640ea43 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:06.561018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d459a-1de7-4ce1-8281-edfde640ea43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:06.562925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6f3d459a-1de7-4ce1-8281-edfde640ea43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:06.643667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d459a-1de7-4ce1-8281-edfde640ea43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:06.645965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d459a-1de7-4ce1-8281-edfde640ea43 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:06.679585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1029/2066] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:06.690513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d426716c2e446da702c718fab1d243'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.690882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d426716c2e446da702c718fab1d243'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.692127 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d426716c2e446da702c718fab1d243'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.702953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d426716c2e446da702c718fab1d243'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.720997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d426716c2e446da702c718fab1d243'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:06.868045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1038/2067] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:06 2022] DELETE /identity/v3/users/39d426716c2e446da702c718fab1d243 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:06.929377 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d459a-1de7-4ce1-8281-edfde640ea43 service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:06.931039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6f3d459a-1de7-4ce1-8281-edfde640ea43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:06.956274 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d459a-1de7-4ce1-8281-edfde640ea43 service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:07.048480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:07.049276 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:07.049539 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:07.049819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] PATH_INFO: `/v3/projects/11f303059bc34edfba1b471d8e862e23` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:07.051606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f303059bc34edfba1b471d8e862e23)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:07.053561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:07.208690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f303059bc34edfba1b471d8e862e23', 'request_id': 'req-24312385-1968-4fdf-9ce7-8ec49b983c9e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.218606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f303059bc34edfba1b471d8e862e23', 'request_id': 'req-24312385-1968-4fdf-9ce7-8ec49b983c9e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.221665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f303059bc34edfba1b471d8e862e23', 'request_id': 'req-24312385-1968-4fdf-9ce7-8ec49b983c9e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.267069 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:07.351330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] The token cache is being invalidate because project 11f303059bc34edfba1b471d8e862e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:07.355739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.355739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.355739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1030/2068] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:06 2022] DELETE /identity/v3/projects/11f303059bc34edfba1b471d8e862e23 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:07.357816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:07.359270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:07.373005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8a5e0-c671-421c-b22d-e50c1005bb8e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:07.439286 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:07.441355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:07.489394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:07.495138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:07.495138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:07.495138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] PATH_INFO: `/v3/users/daba0bb0ec3d4576be063180430c8bef` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:07.495138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] RBAC: Authorizing `identity:delete_user(user_id=daba0bb0ec3d4576be063180430c8bef)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:07.497296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:07.550820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24312385-1968-4fdf-9ce7-8ec49b983c9e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:07.556127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:07.557031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:07.557315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:07.557604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] PATH_INFO: `/v3/projects/cd01c277390f4c808307cc71b28718d2` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:07.558790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd01c277390f4c808307cc71b28718d2)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:07.560777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:07.694601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daba0bb0ec3d4576be063180430c8bef'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.694815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd01c277390f4c808307cc71b28718d2', 'request_id': 'req-c154dc70-a9a9-4530-ade9-9bffc77678e3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.695234 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daba0bb0ec3d4576be063180430c8bef'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.696031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daba0bb0ec3d4576be063180430c8bef'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.701236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd01c277390f4c808307cc71b28718d2', 'request_id': 'req-c154dc70-a9a9-4530-ade9-9bffc77678e3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.704906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daba0bb0ec3d4576be063180430c8bef'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.708303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd01c277390f4c808307cc71b28718d2', 'request_id': 'req-c154dc70-a9a9-4530-ade9-9bffc77678e3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.720968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-d8136391-96d4-4582-b70c-139143b4439f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daba0bb0ec3d4576be063180430c8bef'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.825771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] The token cache is being invalidate because project cd01c277390f4c808307cc71b28718d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:07.826080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.827326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:07.833468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1031/2069] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:07 2022] DELETE /identity/v3/projects/cd01c277390f4c808307cc71b28718d2 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:07.833468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:07.835421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:07.836964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:07.882970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1039/2070] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:07 2022] DELETE /identity/v3/users/daba0bb0ec3d4576be063180430c8bef => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:07.884713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cde158-89cb-4349-8e34-40e71dcd2bcd None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:07.885182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cde158-89cb-4349-8e34-40e71dcd2bcd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:07.885605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cde158-89cb-4349-8e34-40e71dcd2bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:07.908416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b6cde158-89cb-4349-8e34-40e71dcd2bcd None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:07.909674 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b6cde158-89cb-4349-8e34-40e71dcd2bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:07.918410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154dc70-a9a9-4530-ade9-9bffc77678e3 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:08.001458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1040/2071] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:08.010414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3df23-623a-4fcb-a5c7-346f49bbf412 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.011025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3df23-623a-4fcb-a5c7-346f49bbf412 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.011573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3df23-623a-4fcb-a5c7-346f49bbf412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.034076 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:08.038706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.038706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.038706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] PATH_INFO: `/v3/users/c2c32310b7064390a628d8e190ea6027` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.039571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-bcd3df23-623a-4fcb-a5c7-346f49bbf412 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:08.039694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c32310b7064390a628d8e190ea6027)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:08.039792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bcd3df23-623a-4fcb-a5c7-346f49bbf412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:08.039879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:08.052424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1041/2072] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:08.056052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3df23-623a-4fcb-a5c7-346f49bbf412 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:08.071052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3df23-623a-4fcb-a5c7-346f49bbf412 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:08.077788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac7678-fb41-42fd-a920-155053ef6be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:08.078535 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac7678-fb41-42fd-a920-155053ef6be1 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.078887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac7678-fb41-42fd-a920-155053ef6be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.079226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac7678-fb41-42fd-a920-155053ef6be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.080592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac7678-fb41-42fd-a920-155053ef6be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:08.081386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac7678-fb41-42fd-a920-155053ef6be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:08.083589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac7678-fb41-42fd-a920-155053ef6be1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:08.105153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1042/2073] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:57:08.110701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac7678-fb41-42fd-a920-155053ef6be1 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:08.129853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac7678-fb41-42fd-a920-155053ef6be1 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:08.131139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35285e48-921f-4e14-a57e-fe0daa6dae56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:08.132163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35285e48-921f-4e14-a57e-fe0daa6dae56 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.132484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35285e48-921f-4e14-a57e-fe0daa6dae56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.132771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35285e48-921f-4e14-a57e-fe0daa6dae56 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.133676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35285e48-921f-4e14-a57e-fe0daa6dae56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:08.137201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35285e48-921f-4e14-a57e-fe0daa6dae56 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:08.248097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1043/2074] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:08 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:08.248097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c898d-89f0-4100-9596-441c6f91fba4 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.248097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c898d-89f0-4100-9596-441c6f91fba4 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.248097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c898d-89f0-4100-9596-441c6f91fba4 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.248973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1044/2075] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 15:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 15:57:08.249670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c898d-89f0-4100-9596-441c6f91fba4 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:08.302288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c32310b7064390a628d8e190ea6027'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:08.305880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c32310b7064390a628d8e190ea6027'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:08.342577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c32310b7064390a628d8e190ea6027'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:08.344177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c32310b7064390a628d8e190ea6027'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:08.364543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c32310b7064390a628d8e190ea6027'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:08.393157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c898d-89f0-4100-9596-441c6f91fba4 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:08.395513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bed30-bd75-48ab-9ea3-1342d80e3777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:08.396570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bed30-bd75-48ab-9ea3-1342d80e3777 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.397020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bed30-bd75-48ab-9ea3-1342d80e3777 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.397453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bed30-bd75-48ab-9ea3-1342d80e3777 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.398634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bed30-bd75-48ab-9ea3-1342d80e3777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:08.401115 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bed30-bd75-48ab-9ea3-1342d80e3777 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:08.517710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1032/2076] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:07 2022] DELETE /identity/v3/users/c2c32310b7064390a628d8e190ea6027 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:08.519137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:08.520435 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:08.544812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1045/2077] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:08 2022] POST /identity/v3/users => generated 284 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:08.545703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bed30-bd75-48ab-9ea3-1342d80e3777 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:08.547478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-567bed30-bd75-48ab-9ea3-1342d80e3777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:08.587588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab1d69-e861-4f80-a112-4c43e13d51a8 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:08.594697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:08.595880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.599255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.599564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] PATH_INFO: `/v3/projects/0010d15f7c554420b8f76c04d868becc` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.605376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0010d15f7c554420b8f76c04d868becc)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:08.605376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:08.612737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bed30-bd75-48ab-9ea3-1342d80e3777 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:08.662847 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799cf8a-9801-4663-b862-12a54f92b825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:08.663495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799cf8a-9801-4663-b862-12a54f92b825 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.663786 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799cf8a-9801-4663-b862-12a54f92b825 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.664066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799cf8a-9801-4663-b862-12a54f92b825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.665880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e799cf8a-9801-4663-b862-12a54f92b825 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:08.727015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799cf8a-9801-4663-b862-12a54f92b825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:08.728489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799cf8a-9801-4663-b862-12a54f92b825 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:08.737854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1046/2078] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:08.739662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7b890-9261-4c42-a27e-d6347464fe97 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.739978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7b890-9261-4c42-a27e-d6347464fe97 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.740219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7b890-9261-4c42-a27e-d6347464fe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.752849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0010d15f7c554420b8f76c04d868becc', 'request_id': 'req-7191f79f-2cc6-4251-9483-2fa6126c6f01'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:08.760077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0010d15f7c554420b8f76c04d868becc', 'request_id': 'req-7191f79f-2cc6-4251-9483-2fa6126c6f01'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:08.762366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-59b7b890-9261-4c42-a27e-d6347464fe97 None None] MFA Rules not processed for user `312ed2b555424c6a96b3d972ffea5ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:08.762951 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-59b7b890-9261-4c42-a27e-d6347464fe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:08.766112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0010d15f7c554420b8f76c04d868becc', 'request_id': 'req-7191f79f-2cc6-4251-9483-2fa6126c6f01'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:08.830084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1047/2079] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 15:57:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:08.831493 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec50000-4d64-4ca5-8293-a542369c3daf None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.831782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec50000-4d64-4ca5-8293-a542369c3daf None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.832060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec50000-4d64-4ca5-8293-a542369c3daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.854565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-4ec50000-4d64-4ca5-8293-a542369c3daf None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:08.855778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4ec50000-4d64-4ca5-8293-a542369c3daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:08.859784 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] The token cache is being invalidate because project 0010d15f7c554420b8f76c04d868becc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:08.860076 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:08.860798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:08.866627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1033/2080] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:08 2022] DELETE /identity/v3/projects/0010d15f7c554420b8f76c04d868becc => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:08.866627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:08.866627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:08.869356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:08.933823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1048/2081] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:08.937282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec50000-4d64-4ca5-8293-a542369c3daf None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:08.938100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191f79f-2cc6-4251-9483-2fa6126c6f01 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:08.943463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4ec50000-4d64-4ca5-8293-a542369c3daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:08.943656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31158e1f-ecc5-4453-979e-154a617727b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:08.943955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31158e1f-ecc5-4453-979e-154a617727b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.944272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31158e1f-ecc5-4453-979e-154a617727b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.944585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31158e1f-ecc5-4453-979e-154a617727b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.945326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31158e1f-ecc5-4453-979e-154a617727b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:08.946956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31158e1f-ecc5-4453-979e-154a617727b5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:08.951284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec50000-4d64-4ca5-8293-a542369c3daf None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:08.955849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:08.956741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.957026 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.957330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] PATH_INFO: `/v3/users/c2075a6ceede424087448411badff88b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.957881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1034/2082] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:08.958689 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2075a6ceede424087448411badff88b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:08.958844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31158e1f-ecc5-4453-979e-154a617727b5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:08.960796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-31158e1f-ecc5-4453-979e-154a617727b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:08.967056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:08.975515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31158e1f-ecc5-4453-979e-154a617727b5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:08.980631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a1190-502d-46c5-a2d7-3f67828b2612 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:08.981310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a1190-502d-46c5-a2d7-3f67828b2612 service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:08.981598 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a1190-502d-46c5-a2d7-3f67828b2612 service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:08.982109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a1190-502d-46c5-a2d7-3f67828b2612 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:08.983623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ed3a1190-502d-46c5-a2d7-3f67828b2612 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:08.995672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a1190-502d-46c5-a2d7-3f67828b2612 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:08.997309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a1190-502d-46c5-a2d7-3f67828b2612 service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:09.006958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1035/2083] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:09.008306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa70ea-38f5-4de7-8e1e-59db75729176 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.008621 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa70ea-38f5-4de7-8e1e-59db75729176 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.008901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa70ea-38f5-4de7-8e1e-59db75729176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.029418 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b6fa70ea-38f5-4de7-8e1e-59db75729176 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:09.031241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b6fa70ea-38f5-4de7-8e1e-59db75729176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:09.039831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1036/2084] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:09.040675 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa70ea-38f5-4de7-8e1e-59db75729176 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:09.042113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b6fa70ea-38f5-4de7-8e1e-59db75729176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:09.120524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa70ea-38f5-4de7-8e1e-59db75729176 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:09.124312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe23f3d-6468-4868-96f5-7e930dbdc576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:09.124991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe23f3d-6468-4868-96f5-7e930dbdc576 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.125277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe23f3d-6468-4868-96f5-7e930dbdc576 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.125555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe23f3d-6468-4868-96f5-7e930dbdc576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.134580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe23f3d-6468-4868-96f5-7e930dbdc576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:09.136007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe23f3d-6468-4868-96f5-7e930dbdc576 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:09.145985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1037/2085] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:09.146770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe23f3d-6468-4868-96f5-7e930dbdc576 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:09.155909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe23f3d-6468-4868-96f5-7e930dbdc576 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:09.161422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a54b5b-20f1-40b1-beeb-b07348161701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:09.162078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54b5b-20f1-40b1-beeb-b07348161701 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.162361 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54b5b-20f1-40b1-beeb-b07348161701 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.162635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54b5b-20f1-40b1-beeb-b07348161701 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.163355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a54b5b-20f1-40b1-beeb-b07348161701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:09.164959 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a54b5b-20f1-40b1-beeb-b07348161701 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:09.176822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1038/2086] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:09.178352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a61d1-1b37-44b3-b90e-739db9a7ef82 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.178473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a61d1-1b37-44b3-b90e-739db9a7ef82 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.178656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a61d1-1b37-44b3-b90e-739db9a7ef82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.196849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-609a61d1-1b37-44b3-b90e-739db9a7ef82 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:09.198065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-609a61d1-1b37-44b3-b90e-739db9a7ef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:09.206049 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1039/2087] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:09.206810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a61d1-1b37-44b3-b90e-739db9a7ef82 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:09.219096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a61d1-1b37-44b3-b90e-739db9a7ef82 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:09.224666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20feb1b8-be79-4c92-9a6a-c3ec00cef906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:09.225675 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20feb1b8-be79-4c92-9a6a-c3ec00cef906 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.225964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20feb1b8-be79-4c92-9a6a-c3ec00cef906 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.226246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20feb1b8-be79-4c92-9a6a-c3ec00cef906 admin admin] PATH_INFO: `/v3/projects/536e030e42984f75bd593ce7bb77ce5b/users/14591cf9c9bb408395689602ad66785b/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.241092 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20feb1b8-be79-4c92-9a6a-c3ec00cef906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=536e030e42984f75bd593ce7bb77ce5b, user_id=14591cf9c9bb408395689602ad66785b, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:09.243769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20feb1b8-be79-4c92-9a6a-c3ec00cef906 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:09.279205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2075a6ceede424087448411badff88b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:09.279854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2075a6ceede424087448411badff88b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:09.280518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2075a6ceede424087448411badff88b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:09.288233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2075a6ceede424087448411badff88b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:09.300163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7da02dbb-9e5b-43e6-9929-813417e8bad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2075a6ceede424087448411badff88b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:09.348980 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1040/2088] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:09 2022] PUT /identity/v3/projects/536e030e42984f75bd593ce7bb77ce5b/users/14591cf9c9bb408395689602ad66785b/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:09.350230 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20feb1b8-be79-4c92-9a6a-c3ec00cef906 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:09.351956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-20feb1b8-be79-4c92-9a6a-c3ec00cef906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:09.369957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1049/2089] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:08 2022] DELETE /identity/v3/users/c2075a6ceede424087448411badff88b => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:09.377290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cc22e-5c0c-4b74-89bf-cc87e2a65edf None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.377290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cc22e-5c0c-4b74-89bf-cc87e2a65edf None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.377290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cc22e-5c0c-4b74-89bf-cc87e2a65edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.401344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b24cc22e-5c0c-4b74-89bf-cc87e2a65edf None None] MFA Rules not processed for user `14591cf9c9bb408395689602ad66785b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:09.402591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b24cc22e-5c0c-4b74-89bf-cc87e2a65edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:09.426587 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20feb1b8-be79-4c92-9a6a-c3ec00cef906 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:09.484354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1050/2090] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:09.488793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cc22e-5c0c-4b74-89bf-cc87e2a65edf None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:09.490168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b24cc22e-5c0c-4b74-89bf-cc87e2a65edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:09.495771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529d699-a98d-4938-823d-2d0c3a6a3085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:09.496510 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529d699-a98d-4938-823d-2d0c3a6a3085 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.497076 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529d699-a98d-4938-823d-2d0c3a6a3085 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.497076 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529d699-a98d-4938-823d-2d0c3a6a3085 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.497815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529d699-a98d-4938-823d-2d0c3a6a3085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:09.498128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529d699-a98d-4938-823d-2d0c3a6a3085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:09.499744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529d699-a98d-4938-823d-2d0c3a6a3085 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:09.505598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cc22e-5c0c-4b74-89bf-cc87e2a65edf None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:09.511277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46c703-f5dc-4731-b745-17694c3e4374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:09.512685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46c703-f5dc-4731-b745-17694c3e4374 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.512987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46c703-f5dc-4731-b745-17694c3e4374 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.520753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46c703-f5dc-4731-b745-17694c3e4374 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.520753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46c703-f5dc-4731-b745-17694c3e4374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:09.520753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46c703-f5dc-4731-b745-17694c3e4374 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:09.520992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1041/2091] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:57:09.524181 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529d699-a98d-4938-823d-2d0c3a6a3085 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:09.539012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529d699-a98d-4938-823d-2d0c3a6a3085 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:09.544557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95bb5d-0f02-4dd4-9590-529358f57dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:09.545242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95bb5d-0f02-4dd4-9590-529358f57dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.545521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95bb5d-0f02-4dd4-9590-529358f57dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.545958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95bb5d-0f02-4dd4-9590-529358f57dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.546709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95bb5d-0f02-4dd4-9590-529358f57dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:09.548891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95bb5d-0f02-4dd4-9590-529358f57dc0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:09.634114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1051/2092] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:09 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:09.635222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46c703-f5dc-4731-b745-17694c3e4374 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:09.637175 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7b46c703-f5dc-4731-b745-17694c3e4374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:09.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1042/2093] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:09 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:09.686988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95bb5d-0f02-4dd4-9590-529358f57dc0 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:09.727541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46c703-f5dc-4731-b745-17694c3e4374 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:09.772846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95bb5d-0f02-4dd4-9590-529358f57dc0 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:09.778291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdecef-f107-430a-9872-41dc40c78d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:09.778975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdecef-f107-430a-9872-41dc40c78d58 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.779263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdecef-f107-430a-9872-41dc40c78d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.779545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdecef-f107-430a-9872-41dc40c78d58 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.780243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdecef-f107-430a-9872-41dc40c78d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:09.782452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdecef-f107-430a-9872-41dc40c78d58 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:09.785410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:09.786348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.786632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.786906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] PATH_INFO: `/v3/projects/c1d9de4cc70d4be8a3201bd4efd09351` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.788168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d9de4cc70d4be8a3201bd4efd09351)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:09.792492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:09.907338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1043/2094] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:09 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:09.907338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdecef-f107-430a-9872-41dc40c78d58 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:09.919516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdecef-f107-430a-9872-41dc40c78d58 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:09.931789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d9de4cc70d4be8a3201bd4efd09351', 'request_id': 'req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:09.938875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d9de4cc70d4be8a3201bd4efd09351', 'request_id': 'req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:09.945538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d9de4cc70d4be8a3201bd4efd09351', 'request_id': 'req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:09.977755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f412b4c-1359-4ff5-8c29-93d72bd00ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:09.978426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f412b4c-1359-4ff5-8c29-93d72bd00ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:09.978708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f412b4c-1359-4ff5-8c29-93d72bd00ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:09.978981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f412b4c-1359-4ff5-8c29-93d72bd00ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:09.979668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f412b4c-1359-4ff5-8c29-93d72bd00ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:09.981666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f412b4c-1359-4ff5-8c29-93d72bd00ed1 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:09.995147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:10.152389 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] The token cache is being invalidate because project c1d9de4cc70d4be8a3201bd4efd09351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:10.152634 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1044/2095] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:09 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:10.153359 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.153757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.155601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1052/2096] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:09 2022] DELETE /identity/v3/projects/c1d9de4cc70d4be8a3201bd4efd09351 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:10.156641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f412b4c-1359-4ff5-8c29-93d72bd00ed1 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:10.157786 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:10.159582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0f412b4c-1359-4ff5-8c29-93d72bd00ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:10.159709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:10.172850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f412b4c-1359-4ff5-8c29-93d72bd00ed1 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:10.176656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f89bd-a7fd-4384-820b-ade9cb35dc4b admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:10.178454 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:10.179315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.179610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.179888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] PATH_INFO: `/v3/projects/5acfb4c0798d48dda4afb4cf2ea06ed0` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.180301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b98a42-033e-466a-9f42-51e76f354881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:10.181052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b98a42-033e-466a-9f42-51e76f354881 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.181230 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5acfb4c0798d48dda4afb4cf2ea06ed0)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:10.181328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b98a42-033e-466a-9f42-51e76f354881 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.181607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b98a42-033e-466a-9f42-51e76f354881 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.182464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b98a42-033e-466a-9f42-51e76f354881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:10.183155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:10.184201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b98a42-033e-466a-9f42-51e76f354881 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:10.195700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1053/2097] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:10.197303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b98a42-033e-466a-9f42-51e76f354881 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:10.199443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-99b98a42-033e-466a-9f42-51e76f354881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:10.212385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b98a42-033e-466a-9f42-51e76f354881 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:10.217378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552441d-b1fc-4ae3-a84f-9303a604ebbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:10.218065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552441d-b1fc-4ae3-a84f-9303a604ebbe admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.218354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552441d-b1fc-4ae3-a84f-9303a604ebbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.218637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552441d-b1fc-4ae3-a84f-9303a604ebbe admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.219400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552441d-b1fc-4ae3-a84f-9303a604ebbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:10.221068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552441d-b1fc-4ae3-a84f-9303a604ebbe admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:10.231238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1054/2098] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:10.232041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552441d-b1fc-4ae3-a84f-9303a604ebbe admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:10.235522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3552441d-b1fc-4ae3-a84f-9303a604ebbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:10.251145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552441d-b1fc-4ae3-a84f-9303a604ebbe admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:10.256785 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:10.257656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.257937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.258213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] PATH_INFO: `/v3/users/2be1b77c3942405a9201e80f4268778b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.259398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be1b77c3942405a9201e80f4268778b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:10.261618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:10.315916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5acfb4c0798d48dda4afb4cf2ea06ed0', 'request_id': 'req-696a06b3-2f9a-44ca-9fb0-01866e11a31c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.334498 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5acfb4c0798d48dda4afb4cf2ea06ed0', 'request_id': 'req-696a06b3-2f9a-44ca-9fb0-01866e11a31c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.342156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5acfb4c0798d48dda4afb4cf2ea06ed0', 'request_id': 'req-696a06b3-2f9a-44ca-9fb0-01866e11a31c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.404713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] The token cache is being invalidate because project 5acfb4c0798d48dda4afb4cf2ea06ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:10.405003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.405673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.407328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1045/2099] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:10 2022] DELETE /identity/v3/projects/5acfb4c0798d48dda4afb4cf2ea06ed0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:10.408091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:10.409408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:10.471989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a06b3-2f9a-44ca-9fb0-01866e11a31c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:10.477353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed05e4-fc7e-4c0d-9828-bb6329e0f669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:10.478246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed05e4-fc7e-4c0d-9828-bb6329e0f669 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.478530 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed05e4-fc7e-4c0d-9828-bb6329e0f669 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.478808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed05e4-fc7e-4c0d-9828-bb6329e0f669 admin admin] PATH_INFO: `/v3/projects/3944d023a8ce495aa9213f9ef17b9e1a/users/172f88337a93407f8f3c4a8b05def02b/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.493931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed05e4-fc7e-4c0d-9828-bb6329e0f669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3944d023a8ce495aa9213f9ef17b9e1a, user_id=172f88337a93407f8f3c4a8b05def02b, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:10.496586 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed05e4-fc7e-4c0d-9828-bb6329e0f669 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:10.513928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be1b77c3942405a9201e80f4268778b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.514584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be1b77c3942405a9201e80f4268778b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.515198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be1b77c3942405a9201e80f4268778b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.522371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be1b77c3942405a9201e80f4268778b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.535084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-6c6b5717-fe0c-4708-8277-03ca65ce16f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be1b77c3942405a9201e80f4268778b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:10.585974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1046/2100] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:10 2022] PUT /identity/v3/projects/3944d023a8ce495aa9213f9ef17b9e1a/users/172f88337a93407f8f3c4a8b05def02b/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:10.586938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed05e4-fc7e-4c0d-9828-bb6329e0f669 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:10.588571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3aed05e4-fc7e-4c0d-9828-bb6329e0f669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:10.612765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1055/2101] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:10 2022] DELETE /identity/v3/users/2be1b77c3942405a9201e80f4268778b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:10.616448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f92b-ece4-4d12-bf59-5a0b4e9b099e None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.616448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f92b-ece4-4d12-bf59-5a0b4e9b099e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.616448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f92b-ece4-4d12-bf59-5a0b4e9b099e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.634875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-1b81f92b-ece4-4d12-bf59-5a0b4e9b099e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:10.635562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1b81f92b-ece4-4d12-bf59-5a0b4e9b099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:10.661819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed05e4-fc7e-4c0d-9828-bb6329e0f669 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:10.714248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1056/2102] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:10.714248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f92b-ece4-4d12-bf59-5a0b4e9b099e None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:10.717597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1b81f92b-ece4-4d12-bf59-5a0b4e9b099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:10.739833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f92b-ece4-4d12-bf59-5a0b4e9b099e None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:10.754302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a69bf-057d-4405-805d-f4ec42df860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:10.754302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a69bf-057d-4405-805d-f4ec42df860a admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.754991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a69bf-057d-4405-805d-f4ec42df860a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.755286 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a69bf-057d-4405-805d-f4ec42df860a admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.755839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742e1a2-594d-4499-a051-faf05978024e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:10.756209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a69bf-057d-4405-805d-f4ec42df860a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:10.756556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742e1a2-594d-4499-a051-faf05978024e admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.756842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742e1a2-594d-4499-a051-faf05978024e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.757122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742e1a2-594d-4499-a051-faf05978024e admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.757919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742e1a2-594d-4499-a051-faf05978024e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:10.758021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a69bf-057d-4405-805d-f4ec42df860a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:10.765943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742e1a2-594d-4499-a051-faf05978024e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:10.781987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1047/2103] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:10.781987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed40ac-bc3f-4d4b-95d3-c6b38b5428d8 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.781987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed40ac-bc3f-4d4b-95d3-c6b38b5428d8 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.781987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed40ac-bc3f-4d4b-95d3-c6b38b5428d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.792311 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1057/2104] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:10.796684 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742e1a2-594d-4499-a051-faf05978024e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:10.819569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742e1a2-594d-4499-a051-faf05978024e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:10.826276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565f868-db8c-4bed-a44e-b6ee9395073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:10.827243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565f868-db8c-4bed-a44e-b6ee9395073f admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.827539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565f868-db8c-4bed-a44e-b6ee9395073f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.827860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565f868-db8c-4bed-a44e-b6ee9395073f admin admin] PATH_INFO: `/v3/projects/19a6858b332d4c7cb12a9ae7cb7963b3/users/08a091a4ac1546d6b390089a5bc45da7/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.828560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-3aed40ac-bc3f-4d4b-95d3-c6b38b5428d8 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:10.830272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3aed40ac-bc3f-4d4b-95d3-c6b38b5428d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:10.840730 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1048/2105] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:10.843101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed40ac-bc3f-4d4b-95d3-c6b38b5428d8 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:10.851397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565f868-db8c-4bed-a44e-b6ee9395073f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a6858b332d4c7cb12a9ae7cb7963b3, user_id=08a091a4ac1546d6b390089a5bc45da7, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:10.854247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565f868-db8c-4bed-a44e-b6ee9395073f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:10.856920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed40ac-bc3f-4d4b-95d3-c6b38b5428d8 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:10.858543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:10.862218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a883e-42eb-4d4b-a8bd-4c229b6f91bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:10.867511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a883e-42eb-4d4b-a8bd-4c229b6f91bc admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.867511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a883e-42eb-4d4b-a8bd-4c229b6f91bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.867511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a883e-42eb-4d4b-a8bd-4c229b6f91bc admin admin] PATH_INFO: `/v3/projects/3944d023a8ce495aa9213f9ef17b9e1a/users/172f88337a93407f8f3c4a8b05def02b/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.873971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a883e-42eb-4d4b-a8bd-4c229b6f91bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3944d023a8ce495aa9213f9ef17b9e1a, user_id=172f88337a93407f8f3c4a8b05def02b, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:10.876792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a883e-42eb-4d4b-a8bd-4c229b6f91bc admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:10.888065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1049/2106] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:10 2022] PUT /identity/v3/projects/3944d023a8ce495aa9213f9ef17b9e1a/users/172f88337a93407f8f3c4a8b05def02b/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:10.888993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a883e-42eb-4d4b-a8bd-4c229b6f91bc admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:10.962714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1058/2107] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:10 2022] PUT /identity/v3/projects/19a6858b332d4c7cb12a9ae7cb7963b3/users/08a091a4ac1546d6b390089a5bc45da7/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:10.964380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5117de-aaea-4f04-a97c-14dd237f4a4b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:10.964736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5117de-aaea-4f04-a97c-14dd237f4a4b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:10.965046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5117de-aaea-4f04-a97c-14dd237f4a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:10.992173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a883e-42eb-4d4b-a8bd-4c229b6f91bc admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.011232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0b5117de-aaea-4f04-a97c-14dd237f4a4b None None] MFA Rules not processed for user `172f88337a93407f8f3c4a8b05def02b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:11.012834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0b5117de-aaea-4f04-a97c-14dd237f4a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:11.073473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c46e28-1a92-4ab2-b6c9-4122c50cc579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.074187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c46e28-1a92-4ab2-b6c9-4122c50cc579 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.074511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c46e28-1a92-4ab2-b6c9-4122c50cc579 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.074740 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c46e28-1a92-4ab2-b6c9-4122c50cc579 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.075548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c46e28-1a92-4ab2-b6c9-4122c50cc579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:11.075867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c46e28-1a92-4ab2-b6c9-4122c50cc579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.077786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c46e28-1a92-4ab2-b6c9-4122c50cc579 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.092349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1050/2108] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:57:11.093837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98b169-7495-486f-9095-4933b066e1a2 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.094129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98b169-7495-486f-9095-4933b066e1a2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.094411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98b169-7495-486f-9095-4933b066e1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.107116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1059/2109] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:10 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:11.108549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5117de-aaea-4f04-a97c-14dd237f4a4b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.129205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5117de-aaea-4f04-a97c-14dd237f4a4b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.133944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b216c9a-22a7-45ef-8c97-ee491d81b870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.137397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b216c9a-22a7-45ef-8c97-ee491d81b870 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.137397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b216c9a-22a7-45ef-8c97-ee491d81b870 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.137397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b216c9a-22a7-45ef-8c97-ee491d81b870 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.137397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b216c9a-22a7-45ef-8c97-ee491d81b870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.137798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b216c9a-22a7-45ef-8c97-ee491d81b870 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.142669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9b98b169-7495-486f-9095-4933b066e1a2 None None] MFA Rules not processed for user `08a091a4ac1546d6b390089a5bc45da7`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:11.143897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9b98b169-7495-486f-9095-4933b066e1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:11.207450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1051/2110] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:11.208287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98b169-7495-486f-9095-4933b066e1a2 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.209681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9b98b169-7495-486f-9095-4933b066e1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:11.272619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98b169-7495-486f-9095-4933b066e1a2 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.272826 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1060/2111] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:11 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:11.273925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b216c9a-22a7-45ef-8c97-ee491d81b870 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.277447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0b216c9a-22a7-45ef-8c97-ee491d81b870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:11.326071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e127d9-9101-4c8c-9360-958726001af3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.326822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e127d9-9101-4c8c-9360-958726001af3 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.327120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e127d9-9101-4c8c-9360-958726001af3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.327402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e127d9-9101-4c8c-9360-958726001af3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.337259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b216c9a-22a7-45ef-8c97-ee491d81b870 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.342956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67eb7b-e547-4bb1-9215-fa5f959ebb27 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '142e6e44d6544f6a891ed3f913de74ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.343613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67eb7b-e547-4bb1-9215-fa5f959ebb27 service glance] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.343901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67eb7b-e547-4bb1-9215-fa5f959ebb27 service glance] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.344186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67eb7b-e547-4bb1-9215-fa5f959ebb27 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.391518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e127d9-9101-4c8c-9360-958726001af3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.393110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e127d9-9101-4c8c-9360-958726001af3 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.403924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67eb7b-e547-4bb1-9215-fa5f959ebb27 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.405246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1052/2112] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:11 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:11.406448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e127d9-9101-4c8c-9360-958726001af3 service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.407516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d4e127d9-9101-4c8c-9360-958726001af3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:11.415133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67eb7b-e547-4bb1-9215-fa5f959ebb27 service glance] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.425499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1061/2113] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:11.426285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67eb7b-e547-4bb1-9215-fa5f959ebb27 service glance] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.490223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e127d9-9101-4c8c-9360-958726001af3 service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.505278 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.506258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.506545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.506876 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] PATH_INFO: `/v3/projects/92bb0361afd84fbabea16ecb91b1d57c` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.508165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92bb0361afd84fbabea16ecb91b1d57c)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.511252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.514969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67eb7b-e547-4bb1-9215-fa5f959ebb27 service glance] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.520428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3c960-1650-4c90-911c-05b4760f7633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.521063 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3c960-1650-4c90-911c-05b4760f7633 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.521354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3c960-1650-4c90-911c-05b4760f7633 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.521637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3c960-1650-4c90-911c-05b4760f7633 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.522582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3c960-1650-4c90-911c-05b4760f7633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.524833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3c960-1650-4c90-911c-05b4760f7633 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.647278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1062/2114] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:11 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:11.647278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3c960-1650-4c90-911c-05b4760f7633 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.657074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3c960-1650-4c90-911c-05b4760f7633 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.662575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92bb0361afd84fbabea16ecb91b1d57c', 'request_id': 'req-8418482a-213d-4888-bba2-5c391a208f76'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:11.670829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92bb0361afd84fbabea16ecb91b1d57c', 'request_id': 'req-8418482a-213d-4888-bba2-5c391a208f76'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:11.677825 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92bb0361afd84fbabea16ecb91b1d57c', 'request_id': 'req-8418482a-213d-4888-bba2-5c391a208f76'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:11.711285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993b20a-80d9-4724-91dd-6de8e91160cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.711946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993b20a-80d9-4724-91dd-6de8e91160cd service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.712226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993b20a-80d9-4724-91dd-6de8e91160cd service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.712531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993b20a-80d9-4724-91dd-6de8e91160cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.715196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e993b20a-80d9-4724-91dd-6de8e91160cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:11.740441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] The token cache is being invalidate because project 92bb0361afd84fbabea16ecb91b1d57c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:11.740804 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:11.741468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:11.743261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1053/2115] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:11 2022] DELETE /identity/v3/projects/92bb0361afd84fbabea16ecb91b1d57c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:11.747166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.748585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:11.783328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993b20a-80d9-4724-91dd-6de8e91160cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.784867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993b20a-80d9-4724-91dd-6de8e91160cd service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.787294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:11.794850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1063/2116] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:11.796170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c10c1a-7989-4c19-8626-b41a538f1c0a None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.796643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c10c1a-7989-4c19-8626-b41a538f1c0a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.796860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c10c1a-7989-4c19-8626-b41a538f1c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.823634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e0c10c1a-7989-4c19-8626-b41a538f1c0a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:11.825452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e0c10c1a-7989-4c19-8626-b41a538f1c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:11.838390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418482a-213d-4888-bba2-5c391a208f76 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.849378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1064/2117] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:11.849555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9702c-6c51-4473-a9fe-455c752d08f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.849555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9702c-6c51-4473-a9fe-455c752d08f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.850368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9702c-6c51-4473-a9fe-455c752d08f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.852285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd980c-fc84-44ef-a990-b4ec1aa98eac None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.855864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd980c-fc84-44ef-a990-b4ec1aa98eac None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.856402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd980c-fc84-44ef-a990-b4ec1aa98eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.856910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9702c-6c51-4473-a9fe-455c752d08f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.857996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9702c-6c51-4473-a9fe-455c752d08f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.859810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9702c-6c51-4473-a9fe-455c752d08f7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.877842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1054/2118] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:11.884501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9702c-6c51-4473-a9fe-455c752d08f7 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.884777 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ddbd980c-fc84-44ef-a990-b4ec1aa98eac None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:11.889695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ddbd980c-fc84-44ef-a990-b4ec1aa98eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:11.895717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1065/2119] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:11.900320 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd980c-fc84-44ef-a990-b4ec1aa98eac None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.901555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9702c-6c51-4473-a9fe-455c752d08f7 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.909299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6a959-a71a-44e0-9a7b-1cdda892f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.910297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd980c-fc84-44ef-a990-b4ec1aa98eac None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.910470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6a959-a71a-44e0-9a7b-1cdda892f734 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.910876 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6a959-a71a-44e0-9a7b-1cdda892f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.911316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6a959-a71a-44e0-9a7b-1cdda892f734 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.912595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6a959-a71a-44e0-9a7b-1cdda892f734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.915736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6a959-a71a-44e0-9a7b-1cdda892f734 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.916192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9614d-6858-499c-a54c-166bcffa151f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.916985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9614d-6858-499c-a54c-166bcffa151f admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.917154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9614d-6858-499c-a54c-166bcffa151f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.918233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9614d-6858-499c-a54c-166bcffa151f admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.918520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9614d-6858-499c-a54c-166bcffa151f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:11.918948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9614d-6858-499c-a54c-166bcffa151f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.920656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9614d-6858-499c-a54c-166bcffa151f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.932875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1066/2120] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:57:11.933986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1055/2121] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:11.938958 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9614d-6858-499c-a54c-166bcffa151f admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.942967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6a959-a71a-44e0-9a7b-1cdda892f734 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:11.953372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9614d-6858-499c-a54c-166bcffa151f admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.953673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6a959-a71a-44e0-9a7b-1cdda892f734 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:11.958634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d151a71-2b43-4774-91d7-63c9237e21db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.959298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d151a71-2b43-4774-91d7-63c9237e21db admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.959616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178228e-f1aa-4b9e-b334-83ad6233d097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:11.959743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d151a71-2b43-4774-91d7-63c9237e21db admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.959922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d151a71-2b43-4774-91d7-63c9237e21db admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.960634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d151a71-2b43-4774-91d7-63c9237e21db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.960738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178228e-f1aa-4b9e-b334-83ad6233d097 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:11.960964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178228e-f1aa-4b9e-b334-83ad6233d097 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:11.961240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178228e-f1aa-4b9e-b334-83ad6233d097 admin admin] PATH_INFO: `/v3/projects/be6b7ec2b25447ff9ecc0de3211871ea/users/1d5210cf83e4499a896fb3fb1f705fb6/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:11.962542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d151a71-2b43-4774-91d7-63c9237e21db admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.977235 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178228e-f1aa-4b9e-b334-83ad6233d097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6b7ec2b25447ff9ecc0de3211871ea, user_id=1d5210cf83e4499a896fb3fb1f705fb6, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:11.979934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178228e-f1aa-4b9e-b334-83ad6233d097 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:11.984565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:12.059582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1056/2122] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:11 2022] PUT /identity/v3/projects/be6b7ec2b25447ff9ecc0de3211871ea/users/1d5210cf83e4499a896fb3fb1f705fb6/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:12.064601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1067/2123] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:11 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:12.065872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178228e-f1aa-4b9e-b334-83ad6233d097 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:12.071364 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec07c86-5698-4328-80ba-734373b5b220 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:12.071628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec07c86-5698-4328-80ba-734373b5b220 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:12.071906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec07c86-5698-4328-80ba-734373b5b220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:12.101297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-2ec07c86-5698-4328-80ba-734373b5b220 None None] MFA Rules not processed for user `1d5210cf83e4499a896fb3fb1f705fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:12.102655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2ec07c86-5698-4328-80ba-734373b5b220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:12.152551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178228e-f1aa-4b9e-b334-83ad6233d097 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:12.183517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1068/2124] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:12.198102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec07c86-5698-4328-80ba-734373b5b220 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:12.199833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2ec07c86-5698-4328-80ba-734373b5b220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:12.218901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d95c99-461d-489c-82d5-7d9e23e782e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:12.221447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d95c99-461d-489c-82d5-7d9e23e782e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:12.221447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d95c99-461d-489c-82d5-7d9e23e782e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:12.221447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d95c99-461d-489c-82d5-7d9e23e782e5 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:12.221447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d95c99-461d-489c-82d5-7d9e23e782e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:12.222806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d95c99-461d-489c-82d5-7d9e23e782e5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:12.285511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec07c86-5698-4328-80ba-734373b5b220 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:12.310055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1057/2125] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:12 2022] POST /identity/v3/users => generated 283 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:12.311095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d95c99-461d-489c-82d5-7d9e23e782e5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:12.313482 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e76d5-e5f4-4d79-8893-a9e13ef34212 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:12.314158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e76d5-e5f4-4d79-8893-a9e13ef34212 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:12.314440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e76d5-e5f4-4d79-8893-a9e13ef34212 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:12.314750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e76d5-e5f4-4d79-8893-a9e13ef34212 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:12.322087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d95c99-461d-489c-82d5-7d9e23e782e5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:12.324536 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e76d5-e5f4-4d79-8893-a9e13ef34212 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:12.326139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e76d5-e5f4-4d79-8893-a9e13ef34212 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:12.327319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdd9ee-056f-4b6a-b92b-aaac7041dbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:12.328113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdd9ee-056f-4b6a-b92b-aaac7041dbbc admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:12.328421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdd9ee-056f-4b6a-b92b-aaac7041dbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:12.334420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdd9ee-056f-4b6a-b92b-aaac7041dbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:12.334420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdd9ee-056f-4b6a-b92b-aaac7041dbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:12.334420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdd9ee-056f-4b6a-b92b-aaac7041dbbc admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:12.341634 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1058/2126] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:12.345359 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdd9ee-056f-4b6a-b92b-aaac7041dbbc admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:12.346639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1069/2127] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:12.357869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdd9ee-056f-4b6a-b92b-aaac7041dbbc admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:12.367934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495ff0a-9814-4af9-b565-813f2be15b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:12.368691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495ff0a-9814-4af9-b565-813f2be15b42 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:12.368978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495ff0a-9814-4af9-b565-813f2be15b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:12.369627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495ff0a-9814-4af9-b565-813f2be15b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:12.370221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495ff0a-9814-4af9-b565-813f2be15b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:12.373008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495ff0a-9814-4af9-b565-813f2be15b42 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:12.394807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1059/2128] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:12.400429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e76d5-e5f4-4d79-8893-a9e13ef34212 service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:12.415040 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e76d5-e5f4-4d79-8893-a9e13ef34212 service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:12.419814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d0a36-34c4-4b0a-9c25-bbcfcaa45980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:12.420800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d0a36-34c4-4b0a-9c25-bbcfcaa45980 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:12.421090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d0a36-34c4-4b0a-9c25-bbcfcaa45980 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:12.422241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d0a36-34c4-4b0a-9c25-bbcfcaa45980 admin admin] PATH_INFO: `/v3/projects/168b2b67edb944d49fe4be84ec794d0a/users/7af1a1770a8340d1b8a9df25cee1afe5/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:12.439027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d0a36-34c4-4b0a-9c25-bbcfcaa45980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=168b2b67edb944d49fe4be84ec794d0a, user_id=7af1a1770a8340d1b8a9df25cee1afe5, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:12.442380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d0a36-34c4-4b0a-9c25-bbcfcaa45980 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:12.536882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1070/2129] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:12 2022] PUT /identity/v3/projects/168b2b67edb944d49fe4be84ec794d0a/users/7af1a1770a8340d1b8a9df25cee1afe5/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:12.550227 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88059df-a2a5-4aa2-b6f7-cfad6a4df259 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:12.550523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88059df-a2a5-4aa2-b6f7-cfad6a4df259 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:12.550805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88059df-a2a5-4aa2-b6f7-cfad6a4df259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:12.585035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a88059df-a2a5-4aa2-b6f7-cfad6a4df259 None None] MFA Rules not processed for user `7af1a1770a8340d1b8a9df25cee1afe5`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:12.586056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a88059df-a2a5-4aa2-b6f7-cfad6a4df259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:12.647327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d0a36-34c4-4b0a-9c25-bbcfcaa45980 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:12.736323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1060/2130] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:12.738014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d0a36-34c4-4b0a-9c25-bbcfcaa45980 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:12.744066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:12.745203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:12.745436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:12.745748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] PATH_INFO: `/v3/users/1d5210cf83e4499a896fb3fb1f705fb6` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:12.746819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88059df-a2a5-4aa2-b6f7-cfad6a4df259 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:12.747195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d5210cf83e4499a896fb3fb1f705fb6)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:12.749485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:12.765224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88059df-a2a5-4aa2-b6f7-cfad6a4df259 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:12.771778 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff241277-b16e-4488-9a57-f1db41489f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:12.772543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff241277-b16e-4488-9a57-f1db41489f12 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:12.772836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff241277-b16e-4488-9a57-f1db41489f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:12.773646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff241277-b16e-4488-9a57-f1db41489f12 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:12.774368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff241277-b16e-4488-9a57-f1db41489f12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:12.776900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff241277-b16e-4488-9a57-f1db41489f12 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:12.919125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1061/2131] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:12 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:12.922985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff241277-b16e-4488-9a57-f1db41489f12 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:12.987168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff241277-b16e-4488-9a57-f1db41489f12 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:12.992497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d00f4-1bfd-42da-b8dc-ddf1c25ec1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:12.993174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d00f4-1bfd-42da-b8dc-ddf1c25ec1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:12.993479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d00f4-1bfd-42da-b8dc-ddf1c25ec1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:12.993826 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d00f4-1bfd-42da-b8dc-ddf1c25ec1b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:12.994519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d00f4-1bfd-42da-b8dc-ddf1c25ec1b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:12.996499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d00f4-1bfd-42da-b8dc-ddf1c25ec1b7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:13.096240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1062/2132] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:12 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:13.097603 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d00f4-1bfd-42da-b8dc-ddf1c25ec1b7 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:13.102358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d5210cf83e4499a896fb3fb1f705fb6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:13.103072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d5210cf83e4499a896fb3fb1f705fb6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:13.104593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d5210cf83e4499a896fb3fb1f705fb6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:13.110625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d00f4-1bfd-42da-b8dc-ddf1c25ec1b7 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:13.114893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d5210cf83e4499a896fb3fb1f705fb6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:13.119065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5994c-08d4-47c8-a55e-f9825ab040b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:13.119934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5994c-08d4-47c8-a55e-f9825ab040b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:13.125765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5994c-08d4-47c8-a55e-f9825ab040b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:13.125765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5994c-08d4-47c8-a55e-f9825ab040b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:13.125765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5994c-08d4-47c8-a55e-f9825ab040b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:13.125765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5994c-08d4-47c8-a55e-f9825ab040b1 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:13.130386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-83dc2ef2-9161-4bb2-9098-b5271a03701c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d5210cf83e4499a896fb3fb1f705fb6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:13.144433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1063/2133] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:13.146022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5994c-08d4-47c8-a55e-f9825ab040b1 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:13.153472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:13.160100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2da5994c-08d4-47c8-a55e-f9825ab040b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:13.169115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5994c-08d4-47c8-a55e-f9825ab040b1 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:13.174188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75c41f-c47d-4636-b3f0-34285df002dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:13.175003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75c41f-c47d-4636-b3f0-34285df002dd admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:13.175291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75c41f-c47d-4636-b3f0-34285df002dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:13.175586 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75c41f-c47d-4636-b3f0-34285df002dd admin admin] PATH_INFO: `/v3/projects/f9cea69a92c646f5966d5889275bbe6e/users/f273762bf19042c68a11181894ec5b5c/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:13.193750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75c41f-c47d-4636-b3f0-34285df002dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cea69a92c646f5966d5889275bbe6e, user_id=f273762bf19042c68a11181894ec5b5c, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:13.196423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75c41f-c47d-4636-b3f0-34285df002dd admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:13.219029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1071/2134] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:12 2022] DELETE /identity/v3/users/1d5210cf83e4499a896fb3fb1f705fb6 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:13.224387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37eaed-9a59-4e8a-8906-656f5de182bb None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:13.224679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37eaed-9a59-4e8a-8906-656f5de182bb None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:13.224956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37eaed-9a59-4e8a-8906-656f5de182bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:13.248904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-cc37eaed-9a59-4e8a-8906-656f5de182bb None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:13.250211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-cc37eaed-9a59-4e8a-8906-656f5de182bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:13.303077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1064/2135] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:13 2022] PUT /identity/v3/projects/f9cea69a92c646f5966d5889275bbe6e/users/f273762bf19042c68a11181894ec5b5c/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:13.304075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:13.306973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75c41f-c47d-4636-b3f0-34285df002dd admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:13.367896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1072/2136] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:13.375043 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37eaed-9a59-4e8a-8906-656f5de182bb None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:13.376612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-cc37eaed-9a59-4e8a-8906-656f5de182bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:13.462317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75c41f-c47d-4636-b3f0-34285df002dd admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:13.479949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5a922-eeb1-4dd1-ae66-12b649fdcc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:13.481008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5a922-eeb1-4dd1-ae66-12b649fdcc8a admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:13.481449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5a922-eeb1-4dd1-ae66-12b649fdcc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:13.482107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5a922-eeb1-4dd1-ae66-12b649fdcc8a admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:13.483538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5a922-eeb1-4dd1-ae66-12b649fdcc8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:13.486083 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5a922-eeb1-4dd1-ae66-12b649fdcc8a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:13.502726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37eaed-9a59-4e8a-8906-656f5de182bb None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:13.503829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1065/2137] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:13.504809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5a922-eeb1-4dd1-ae66-12b649fdcc8a admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:13.517190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5a922-eeb1-4dd1-ae66-12b649fdcc8a admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:13.523453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6046ea-defe-4047-87ee-645fc4dcf916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:13.524649 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6046ea-defe-4047-87ee-645fc4dcf916 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:13.524936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6046ea-defe-4047-87ee-645fc4dcf916 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:13.525218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6046ea-defe-4047-87ee-645fc4dcf916 admin admin] PATH_INFO: `/v3/projects/f9cea69a92c646f5966d5889275bbe6e/users/f273762bf19042c68a11181894ec5b5c/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:13.527514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6046ea-defe-4047-87ee-645fc4dcf916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9cea69a92c646f5966d5889275bbe6e, user_id=f273762bf19042c68a11181894ec5b5c, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:13.539999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6046ea-defe-4047-87ee-645fc4dcf916 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:13.566958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1066/2138] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:13 2022] PUT /identity/v3/projects/f9cea69a92c646f5966d5889275bbe6e/users/f273762bf19042c68a11181894ec5b5c/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:13.566958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4b53c-453a-4c8c-a45f-8b8fee8669f0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:13.566958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4b53c-453a-4c8c-a45f-8b8fee8669f0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:13.566958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4b53c-453a-4c8c-a45f-8b8fee8669f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:13.576227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e68320-d771-40e0-957e-53d24137e130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:13.577169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e68320-d771-40e0-957e-53d24137e130 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:13.577402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e68320-d771-40e0-957e-53d24137e130 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:13.577705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e68320-d771-40e0-957e-53d24137e130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:13.591423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e68320-d771-40e0-957e-53d24137e130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:13.593103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e68320-d771-40e0-957e-53d24137e130 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:13.600975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-64f4b53c-453a-4c8c-a45f-8b8fee8669f0 None None] MFA Rules not processed for user `f273762bf19042c68a11181894ec5b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:13.603421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-64f4b53c-453a-4c8c-a45f-8b8fee8669f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:13.617132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1073/2139] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:13.732136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1067/2140] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:13.743532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e68320-d771-40e0-957e-53d24137e130 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:13.745076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-89e68320-d771-40e0-957e-53d24137e130 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:13.792818 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:13.807556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e68320-d771-40e0-957e-53d24137e130 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:13.874092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe0f75-e606-4208-9f9d-ed54eef99afa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:13.874771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe0f75-e606-4208-9f9d-ed54eef99afa service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:13.875056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe0f75-e606-4208-9f9d-ed54eef99afa service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:13.875321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe0f75-e606-4208-9f9d-ed54eef99afa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:13.887743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe0f75-e606-4208-9f9d-ed54eef99afa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:13.895576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe0f75-e606-4208-9f9d-ed54eef99afa service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:13.901259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1074/2141] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:13.990752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:14.109726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4b53c-453a-4c8c-a45f-8b8fee8669f0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:14.134977 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4b53c-453a-4c8c-a45f-8b8fee8669f0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:14.141993 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1aa95-c2ed-480c-bfa8-077303cef5b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:14.142985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1aa95-c2ed-480c-bfa8-077303cef5b1 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:14.143425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1aa95-c2ed-480c-bfa8-077303cef5b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:14.143917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1aa95-c2ed-480c-bfa8-077303cef5b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:14.146806 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-15e1aa95-c2ed-480c-bfa8-077303cef5b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:14.208149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe0f75-e606-4208-9f9d-ed54eef99afa service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:14.209983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-49fe0f75-e606-4208-9f9d-ed54eef99afa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:14.223756 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1aa95-c2ed-480c-bfa8-077303cef5b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:14.225240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1aa95-c2ed-480c-bfa8-077303cef5b1 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:14.248206 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1068/2142] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:14.291273 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe0f75-e606-4208-9f9d-ed54eef99afa service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:14.296034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:14.297088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:14.297379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:14.297660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] PATH_INFO: `/v3/projects/be6b7ec2b25447ff9ecc0de3211871ea` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:14.298853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be6b7ec2b25447ff9ecc0de3211871ea)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:14.300997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:14.379736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1aa95-c2ed-480c-bfa8-077303cef5b1 service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:14.381408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-15e1aa95-c2ed-480c-bfa8-077303cef5b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:14.399075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1aa95-c2ed-480c-bfa8-077303cef5b1 service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:14.401541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:14.408078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:14.409007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:14.409292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:14.409573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] PATH_INFO: `/v3/users/d35e390ae2ad42a4991ae622442cb381` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:14.411536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35e390ae2ad42a4991ae622442cb381)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:14.414011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:14.420428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be6b7ec2b25447ff9ecc0de3211871ea', 'request_id': 'req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.426515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be6b7ec2b25447ff9ecc0de3211871ea', 'request_id': 'req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.432325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be6b7ec2b25447ff9ecc0de3211871ea', 'request_id': 'req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.521545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] The token cache is being invalidate because project be6b7ec2b25447ff9ecc0de3211871ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:14.522063 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.522691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f522ea17-05ee-4e9b-9304-e5cd8f2537d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.524945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1075/2143] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:14 2022] DELETE /identity/v3/projects/be6b7ec2b25447ff9ecc0de3211871ea => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:14.529699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f94c5-4a87-4153-9dd8-0fc1b9b97edc None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:14.529990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f94c5-4a87-4153-9dd8-0fc1b9b97edc None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:14.530429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f94c5-4a87-4153-9dd8-0fc1b9b97edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:14.550995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ae6f94c5-4a87-4153-9dd8-0fc1b9b97edc None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:14.552054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ae6f94c5-4a87-4153-9dd8-0fc1b9b97edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:14.621374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1076/2144] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:14.625328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f94c5-4a87-4153-9dd8-0fc1b9b97edc None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:14.626651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ae6f94c5-4a87-4153-9dd8-0fc1b9b97edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:14.638890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f94c5-4a87-4153-9dd8-0fc1b9b97edc None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:14.644538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:14.645859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:14.646140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:14.646413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] PATH_INFO: `/v3/users/90ce6ce6421c4beb973af5d439e69351` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:14.647638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90ce6ce6421c4beb973af5d439e69351)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:14.649869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:14.656036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35e390ae2ad42a4991ae622442cb381'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.656902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35e390ae2ad42a4991ae622442cb381'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.685440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35e390ae2ad42a4991ae622442cb381'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.686428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35e390ae2ad42a4991ae622442cb381'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.695051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1391da19-a9df-4c71-a4aa-f1a8468ec79c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35e390ae2ad42a4991ae622442cb381'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.786597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1069/2145] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:14 2022] DELETE /identity/v3/users/d35e390ae2ad42a4991ae622442cb381 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:14.787985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977e769-fa98-480f-9b88-66963c1ffdb5 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:14.788277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977e769-fa98-480f-9b88-66963c1ffdb5 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:14.788747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977e769-fa98-480f-9b88-66963c1ffdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:14.809399 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-0977e769-fa98-480f-9b88-66963c1ffdb5 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:14.810618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0977e769-fa98-480f-9b88-66963c1ffdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:14.863963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90ce6ce6421c4beb973af5d439e69351'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.864831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90ce6ce6421c4beb973af5d439e69351'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.865473 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90ce6ce6421c4beb973af5d439e69351'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.875303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1070/2146] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:14.875560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90ce6ce6421c4beb973af5d439e69351'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.876505 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977e769-fa98-480f-9b88-66963c1ffdb5 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:14.886874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977e769-fa98-480f-9b88-66963c1ffdb5 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:14.888907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-101f93dc-3f80-4e10-93e7-20c32a4ae7c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90ce6ce6421c4beb973af5d439e69351'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:14.892266 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b89533-7229-4f7f-9026-8439733f2181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:14.893005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b89533-7229-4f7f-9026-8439733f2181 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:14.893254 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b89533-7229-4f7f-9026-8439733f2181 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:14.893534 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b89533-7229-4f7f-9026-8439733f2181 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:14.894290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b89533-7229-4f7f-9026-8439733f2181 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:14.894596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b89533-7229-4f7f-9026-8439733f2181 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:14.896827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b89533-7229-4f7f-9026-8439733f2181 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:14.909786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1071/2147] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:57:14.914701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b89533-7229-4f7f-9026-8439733f2181 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:14.924926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b89533-7229-4f7f-9026-8439733f2181 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:14.929546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61674aa9-b527-4877-b3ca-becfa24eefc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:14.930215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61674aa9-b527-4877-b3ca-becfa24eefc6 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:14.930498 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61674aa9-b527-4877-b3ca-becfa24eefc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:14.930799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61674aa9-b527-4877-b3ca-becfa24eefc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:14.932496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61674aa9-b527-4877-b3ca-becfa24eefc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:14.933392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61674aa9-b527-4877-b3ca-becfa24eefc6 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:14.984244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1077/2148] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:14 2022] DELETE /identity/v3/users/90ce6ce6421c4beb973af5d439e69351 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:14.988823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c42d0-3af7-4d37-a48e-528a4c0a8332 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:14.989119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c42d0-3af7-4d37-a48e-528a4c0a8332 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:14.989449 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c42d0-3af7-4d37-a48e-528a4c0a8332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:15.007392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-560c42d0-3af7-4d37-a48e-528a4c0a8332 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:15.008532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-560c42d0-3af7-4d37-a48e-528a4c0a8332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:15.064665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1078/2149] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:15.074523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c42d0-3af7-4d37-a48e-528a4c0a8332 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:15.075881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-560c42d0-3af7-4d37-a48e-528a4c0a8332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:15.120859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1072/2150] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:14 2022] POST /identity/v3/projects => generated 371 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:15.122874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61674aa9-b527-4877-b3ca-becfa24eefc6 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:15.126744 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-61674aa9-b527-4877-b3ca-becfa24eefc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:15.137868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c42d0-3af7-4d37-a48e-528a4c0a8332 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:15.190412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504f97c-c6b1-4a66-a1db-8549313b904c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:15.191070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504f97c-c6b1-4a66-a1db-8549313b904c service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:15.191359 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504f97c-c6b1-4a66-a1db-8549313b904c service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:15.191657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504f97c-c6b1-4a66-a1db-8549313b904c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:15.206062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61674aa9-b527-4877-b3ca-becfa24eefc6 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:15.210951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:15.211830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:15.212233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:15.212424 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] PATH_INFO: `/v3/projects/3eb8d4ed271249c0b90da1de7c4f88c3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:15.214563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb8d4ed271249c0b90da1de7c4f88c3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:15.215600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:15.233697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:15.261666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504f97c-c6b1-4a66-a1db-8549313b904c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:15.263179 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504f97c-c6b1-4a66-a1db-8549313b904c service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:15.276163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1079/2151] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:57:15.277190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504f97c-c6b1-4a66-a1db-8549313b904c service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:15.291498 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504f97c-c6b1-4a66-a1db-8549313b904c service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:15.297663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2181-f387-427a-a445-10b92f5256a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:15.298475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2181-f387-427a-a445-10b92f5256a5 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:15.298645 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2181-f387-427a-a445-10b92f5256a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:15.298930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2181-f387-427a-a445-10b92f5256a5 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:15.299641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2181-f387-427a-a445-10b92f5256a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:15.302300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2181-f387-427a-a445-10b92f5256a5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:15.327679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb8d4ed271249c0b90da1de7c4f88c3', 'request_id': 'req-14056f16-2d8f-4e67-947c-de763ca1a5ac'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:15.336178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb8d4ed271249c0b90da1de7c4f88c3', 'request_id': 'req-14056f16-2d8f-4e67-947c-de763ca1a5ac'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:15.342593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb8d4ed271249c0b90da1de7c4f88c3', 'request_id': 'req-14056f16-2d8f-4e67-947c-de763ca1a5ac'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:15.397937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1080/2152] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:15 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:15.401915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2181-f387-427a-a445-10b92f5256a5 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:15.411466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2181-f387-427a-a445-10b92f5256a5 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:15.426909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] The token cache is being invalidate because project 3eb8d4ed271249c0b90da1de7c4f88c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:15.433602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:15.433602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:15.433602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1073/2153] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:15 2022] DELETE /identity/v3/projects/3eb8d4ed271249c0b90da1de7c4f88c3 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:15.435699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:15.437999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:15.485763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335f702-3a14-464e-9b99-fe1527c583ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:15.486818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335f702-3a14-464e-9b99-fe1527c583ec admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:15.487348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335f702-3a14-464e-9b99-fe1527c583ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:15.487912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335f702-3a14-464e-9b99-fe1527c583ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:15.490176 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335f702-3a14-464e-9b99-fe1527c583ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:15.492724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335f702-3a14-464e-9b99-fe1527c583ec admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:15.510193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1081/2154] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:15.511322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335f702-3a14-464e-9b99-fe1527c583ec admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:15.514532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4335f702-3a14-464e-9b99-fe1527c583ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:15.529968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335f702-3a14-464e-9b99-fe1527c583ec admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:15.535307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e805b19-69fd-4855-a782-16d403ebe956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:15.536789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e805b19-69fd-4855-a782-16d403ebe956 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:15.537180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e805b19-69fd-4855-a782-16d403ebe956 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:15.537561 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e805b19-69fd-4855-a782-16d403ebe956 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:15.538599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e805b19-69fd-4855-a782-16d403ebe956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:15.539125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14056f16-2d8f-4e67-947c-de763ca1a5ac admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:15.540613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e805b19-69fd-4855-a782-16d403ebe956 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:15.547869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:15.548932 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:15.549207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:15.549482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] PATH_INFO: `/v3/users/721d786a0c61430e8a51b8debe89e62b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:15.550736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=721d786a0c61430e8a51b8debe89e62b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:15.553447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:15.557834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1082/2155] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:15.558957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e805b19-69fd-4855-a782-16d403ebe956 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:15.582296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e805b19-69fd-4855-a782-16d403ebe956 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:15.588822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c6422-a84e-4b0d-8be2-fd919e25a30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:15.589996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c6422-a84e-4b0d-8be2-fd919e25a30a admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:15.590486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c6422-a84e-4b0d-8be2-fd919e25a30a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:15.590961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c6422-a84e-4b0d-8be2-fd919e25a30a admin admin] PATH_INFO: `/v3/projects/98a5e68ec7ca4f88b875a558beef15c9/users/63b379593c2743999e59c7e23ee27af1/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:15.634614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c6422-a84e-4b0d-8be2-fd919e25a30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a5e68ec7ca4f88b875a558beef15c9, user_id=63b379593c2743999e59c7e23ee27af1, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:15.636437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c6422-a84e-4b0d-8be2-fd919e25a30a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:15.741242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1083/2156] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:15 2022] PUT /identity/v3/projects/98a5e68ec7ca4f88b875a558beef15c9/users/63b379593c2743999e59c7e23ee27af1/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:15.762249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00318259-cb62-4a8b-99de-004e55c379d8 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:15.762249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00318259-cb62-4a8b-99de-004e55c379d8 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:15.762249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00318259-cb62-4a8b-99de-004e55c379d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:15.804452 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-00318259-cb62-4a8b-99de-004e55c379d8 None None] MFA Rules not processed for user `63b379593c2743999e59c7e23ee27af1`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:15.806191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-00318259-cb62-4a8b-99de-004e55c379d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:15.807106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '721d786a0c61430e8a51b8debe89e62b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:15.809147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '721d786a0c61430e8a51b8debe89e62b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:15.853156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:15.853273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '721d786a0c61430e8a51b8debe89e62b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:15.853273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '721d786a0c61430e8a51b8debe89e62b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:15.876795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '721d786a0c61430e8a51b8debe89e62b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:15.940042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1084/2157] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:15.945111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00318259-cb62-4a8b-99de-004e55c379d8 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:15.945111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-00318259-cb62-4a8b-99de-004e55c379d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:15.969845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1074/2158] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:15 2022] DELETE /identity/v3/users/721d786a0c61430e8a51b8debe89e62b => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:15.970885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:15.972230 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:16.055441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00318259-cb62-4a8b-99de-004e55c379d8 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:16.056597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c8561-fbb7-4edb-8448-5a6062fd5a6e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:16.061686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15cb9d-1349-4d3c-9666-7b0c7b8403e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:16.062307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:16.062455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15cb9d-1349-4d3c-9666-7b0c7b8403e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:16.062639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15cb9d-1349-4d3c-9666-7b0c7b8403e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:16.063277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15cb9d-1349-4d3c-9666-7b0c7b8403e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:16.063904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:16.064011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15cb9d-1349-4d3c-9666-7b0c7b8403e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:16.064636 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:16.072988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] PATH_INFO: `/v3/projects/ef23d64217d24ceea3e499abd13976fd` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:16.073420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15cb9d-1349-4d3c-9666-7b0c7b8403e8 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:16.073602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef23d64217d24ceea3e499abd13976fd)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:16.073602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:16.111735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:16.177738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1075/2159] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:15 2022] POST /identity/v3/projects => generated 373 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:16.179026 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15cb9d-1349-4d3c-9666-7b0c7b8403e8 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:16.215982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef23d64217d24ceea3e499abd13976fd', 'request_id': 'req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.222578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef23d64217d24ceea3e499abd13976fd', 'request_id': 'req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.234252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef23d64217d24ceea3e499abd13976fd', 'request_id': 'req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.260639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15cb9d-1349-4d3c-9666-7b0c7b8403e8 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:16.307913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] The token cache is being invalidate because project ef23d64217d24ceea3e499abd13976fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:16.308216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.308929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.311960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1085/2160] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:15 2022] DELETE /identity/v3/projects/ef23d64217d24ceea3e499abd13976fd => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:16.313034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:16.315733 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:16.345998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6015102-7d02-48ed-a7de-267329a8892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:16.346772 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6015102-7d02-48ed-a7de-267329a8892a admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:16.347300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6015102-7d02-48ed-a7de-267329a8892a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:16.348133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6015102-7d02-48ed-a7de-267329a8892a admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:16.350742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6015102-7d02-48ed-a7de-267329a8892a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:16.353246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6015102-7d02-48ed-a7de-267329a8892a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:16.405167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540d431-ca1d-4fcb-8c18-7757a2d2dbb6 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:16.410023 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:16.411162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:16.411570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:16.411956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] PATH_INFO: `/v3/projects/31ac93d3e19d41be9de01b85141bd17d` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:16.413186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ac93d3e19d41be9de01b85141bd17d)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:16.415137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:16.450561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1076/2161] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:16 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:16.451472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6015102-7d02-48ed-a7de-267329a8892a admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:16.456402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b6015102-7d02-48ed-a7de-267329a8892a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:16.465874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6015102-7d02-48ed-a7de-267329a8892a admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:16.470596 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:16.471475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:16.471760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:16.472038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] PATH_INFO: `/v3/users/5faf2fb6460e467b803d6ba452fb03ed` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:16.473394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5faf2fb6460e467b803d6ba452fb03ed)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:16.475438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:16.551409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:16.580134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ac93d3e19d41be9de01b85141bd17d', 'request_id': 'req-8782522c-58ed-409b-b13e-df7124b16b33'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.588868 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ac93d3e19d41be9de01b85141bd17d', 'request_id': 'req-8782522c-58ed-409b-b13e-df7124b16b33'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.594964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ac93d3e19d41be9de01b85141bd17d', 'request_id': 'req-8782522c-58ed-409b-b13e-df7124b16b33'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.672140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5faf2fb6460e467b803d6ba452fb03ed'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.672886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5faf2fb6460e467b803d6ba452fb03ed'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.694579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] The token cache is being invalidate because project 31ac93d3e19d41be9de01b85141bd17d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:16.694760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.695341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.695757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5faf2fb6460e467b803d6ba452fb03ed'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.703793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5faf2fb6460e467b803d6ba452fb03ed'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.703935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1086/2162] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:16 2022] DELETE /identity/v3/projects/31ac93d3e19d41be9de01b85141bd17d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:16.703935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:16.703935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:16.704163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5faf2fb6460e467b803d6ba452fb03ed'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:16.765875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782522c-58ed-409b-b13e-df7124b16b33 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:16.770721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e9091-98af-4f76-9ef1-a19a871a8821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:16.771415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e9091-98af-4f76-9ef1-a19a871a8821 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:16.771706 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e9091-98af-4f76-9ef1-a19a871a8821 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:16.771989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e9091-98af-4f76-9ef1-a19a871a8821 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:16.772750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e9091-98af-4f76-9ef1-a19a871a8821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:16.774679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e9091-98af-4f76-9ef1-a19a871a8821 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:16.788703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1087/2163] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:16.790396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acdfaa-c1b0-459c-9d18-5986956655af None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:16.790756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acdfaa-c1b0-459c-9d18-5986956655af None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:16.790990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acdfaa-c1b0-459c-9d18-5986956655af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:16.798739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1077/2164] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:16 2022] DELETE /identity/v3/users/5faf2fb6460e467b803d6ba452fb03ed => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:16.799602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:16.802444 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:16.820564 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e5acdfaa-c1b0-459c-9d18-5986956655af None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:16.822619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e5acdfaa-c1b0-459c-9d18-5986956655af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:16.901307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc16f2-02f5-43ec-9858-def54dc65ea0 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:16.902481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1088/2165] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:16.907870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94f993-c60a-4f2b-898c-013285c07c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:16.911485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88dfe1-addf-48d2-9fdd-4fa61ef16f76 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:16.911781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88dfe1-addf-48d2-9fdd-4fa61ef16f76 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:16.911926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94f993-c60a-4f2b-898c-013285c07c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:16.912052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88dfe1-addf-48d2-9fdd-4fa61ef16f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:16.912304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94f993-c60a-4f2b-898c-013285c07c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:16.912621 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94f993-c60a-4f2b-898c-013285c07c88 admin admin] PATH_INFO: `/v3/projects/fc7364dd0f2a4ed68b083c9d5e6b1bd1/users/30abcfea60a94267aa3f5c3f0a14ed5a/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:16.931110 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94f993-c60a-4f2b-898c-013285c07c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7364dd0f2a4ed68b083c9d5e6b1bd1, user_id=30abcfea60a94267aa3f5c3f0a14ed5a, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:16.933827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94f993-c60a-4f2b-898c-013285c07c88 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:16.934276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-dc88dfe1-addf-48d2-9fdd-4fa61ef16f76 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:16.935623 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-dc88dfe1-addf-48d2-9fdd-4fa61ef16f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:16.944585 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1089/2166] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:16.948913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88dfe1-addf-48d2-9fdd-4fa61ef16f76 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:16.958395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88dfe1-addf-48d2-9fdd-4fa61ef16f76 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:16.963218 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebeeb06-c44d-4e20-a9e1-076e886fc676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:16.963879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebeeb06-c44d-4e20-a9e1-076e886fc676 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:16.964165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebeeb06-c44d-4e20-a9e1-076e886fc676 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:16.964478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebeeb06-c44d-4e20-a9e1-076e886fc676 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:16.965253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebeeb06-c44d-4e20-a9e1-076e886fc676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:16.965682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebeeb06-c44d-4e20-a9e1-076e886fc676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:16.967286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebeeb06-c44d-4e20-a9e1-076e886fc676 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:16.983259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1090/2167] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:57:16.984328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebeeb06-c44d-4e20-a9e1-076e886fc676 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:16.994733 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebeeb06-c44d-4e20-a9e1-076e886fc676 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:16.999482 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04c852-6077-41f2-b9a3-e99b46265c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:17.000210 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04c852-6077-41f2-b9a3-e99b46265c13 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.000545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04c852-6077-41f2-b9a3-e99b46265c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.000833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04c852-6077-41f2-b9a3-e99b46265c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.001529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04c852-6077-41f2-b9a3-e99b46265c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:17.003509 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04c852-6077-41f2-b9a3-e99b46265c13 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:17.025597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1078/2168] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:16 2022] PUT /identity/v3/projects/fc7364dd0f2a4ed68b083c9d5e6b1bd1/users/30abcfea60a94267aa3f5c3f0a14ed5a/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:17.029094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94f993-c60a-4f2b-898c-013285c07c88 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:17.043939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:17.123942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94f993-c60a-4f2b-898c-013285c07c88 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:17.129458 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99309e-ebc2-4ee4-b273-6349f5ef8510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:17.130140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99309e-ebc2-4ee4-b273-6349f5ef8510 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.130426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99309e-ebc2-4ee4-b273-6349f5ef8510 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.130706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99309e-ebc2-4ee4-b273-6349f5ef8510 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.131445 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99309e-ebc2-4ee4-b273-6349f5ef8510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:17.133098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99309e-ebc2-4ee4-b273-6349f5ef8510 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:17.146289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1079/2169] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:17.148125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99309e-ebc2-4ee4-b273-6349f5ef8510 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:17.158953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99309e-ebc2-4ee4-b273-6349f5ef8510 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:17.165131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328d1e-4cf3-4788-817b-dd8c3b285f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:17.165856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328d1e-4cf3-4788-817b-dd8c3b285f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.166140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328d1e-4cf3-4788-817b-dd8c3b285f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.166414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328d1e-4cf3-4788-817b-dd8c3b285f6d admin admin] PATH_INFO: `/v3/projects/fc7364dd0f2a4ed68b083c9d5e6b1bd1/users/30abcfea60a94267aa3f5c3f0a14ed5a/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.168484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db328d1e-4cf3-4788-817b-dd8c3b285f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7364dd0f2a4ed68b083c9d5e6b1bd1, user_id=30abcfea60a94267aa3f5c3f0a14ed5a, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:17.170645 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1091/2170] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:16 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:17.174291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db328d1e-4cf3-4788-817b-dd8c3b285f6d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:17.174760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04c852-6077-41f2-b9a3-e99b46265c13 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:17.210108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1080/2171] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:17 2022] PUT /identity/v3/projects/fc7364dd0f2a4ed68b083c9d5e6b1bd1/users/30abcfea60a94267aa3f5c3f0a14ed5a/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:17.225657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f9ae3-4419-4292-9482-263186f19374 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.231557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f9ae3-4419-4292-9482-263186f19374 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.231557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f9ae3-4419-4292-9482-263186f19374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.258054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-680f9ae3-4419-4292-9482-263186f19374 None None] MFA Rules not processed for user `30abcfea60a94267aa3f5c3f0a14ed5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:17.259461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-680f9ae3-4419-4292-9482-263186f19374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:17.275394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04c852-6077-41f2-b9a3-e99b46265c13 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:17.336779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1081/2172] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:17.345733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f9ae3-4419-4292-9482-263186f19374 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:17.351524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128156b7-ceb6-4ccf-aa6f-a50a2642bc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:17.352807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128156b7-ceb6-4ccf-aa6f-a50a2642bc74 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.353115 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128156b7-ceb6-4ccf-aa6f-a50a2642bc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.353419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128156b7-ceb6-4ccf-aa6f-a50a2642bc74 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.354217 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-680f9ae3-4419-4292-9482-263186f19374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:17.354436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128156b7-ceb6-4ccf-aa6f-a50a2642bc74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:17.356198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128156b7-ceb6-4ccf-aa6f-a50a2642bc74 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:17.369195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f9ae3-4419-4292-9482-263186f19374 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:17.373933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:17.374788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.375071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.375347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] PATH_INFO: `/v3/projects/0211feb02bc14c428eace423c75dd094` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.376556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0211feb02bc14c428eace423c75dd094)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:17.378480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:17.483582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1092/2173] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:17 2022] POST /identity/v3/users => generated 289 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:17.484435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128156b7-ceb6-4ccf-aa6f-a50a2642bc74 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:17.485978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-128156b7-ceb6-4ccf-aa6f-a50a2642bc74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:17.528308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0211feb02bc14c428eace423c75dd094', 'request_id': 'req-19378fe0-8fe6-4e11-afa6-109321efeecc'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:17.535173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0211feb02bc14c428eace423c75dd094', 'request_id': 'req-19378fe0-8fe6-4e11-afa6-109321efeecc'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:17.541326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0211feb02bc14c428eace423c75dd094', 'request_id': 'req-19378fe0-8fe6-4e11-afa6-109321efeecc'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:17.544524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128156b7-ceb6-4ccf-aa6f-a50a2642bc74 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:17.601632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06102e-ae59-4eb1-8baa-3570f895376a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:17.602279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06102e-ae59-4eb1-8baa-3570f895376a service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.602563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06102e-ae59-4eb1-8baa-3570f895376a service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.602839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06102e-ae59-4eb1-8baa-3570f895376a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.623433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] The token cache is being invalidate because project 0211feb02bc14c428eace423c75dd094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:17.623722 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:17.624372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:17.626211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1082/2174] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:17 2022] DELETE /identity/v3/projects/0211feb02bc14c428eace423c75dd094 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:17.626724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:17.628112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:17.670201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06102e-ae59-4eb1-8baa-3570f895376a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:17.671749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06102e-ae59-4eb1-8baa-3570f895376a service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:17.682359 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1093/2175] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:17.683207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06102e-ae59-4eb1-8baa-3570f895376a service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:17.702145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19378fe0-8fe6-4e11-afa6-109321efeecc admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:17.707870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c9c99-24ac-4242-ab8b-ef44c51981f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:17.716221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c9c99-24ac-4242-ab8b-ef44c51981f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.716221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c9c99-24ac-4242-ab8b-ef44c51981f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.716221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c9c99-24ac-4242-ab8b-ef44c51981f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.716221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c9c99-24ac-4242-ab8b-ef44c51981f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:17.716221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c9c99-24ac-4242-ab8b-ef44c51981f7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:17.724246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1083/2176] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:17.728161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c9c99-24ac-4242-ab8b-ef44c51981f7 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:17.739154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c9c99-24ac-4242-ab8b-ef44c51981f7 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:17.744210 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e86ea6-9d8d-4ec6-9ec7-f81f83850d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:17.744942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e86ea6-9d8d-4ec6-9ec7-f81f83850d25 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.745220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e86ea6-9d8d-4ec6-9ec7-f81f83850d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.745492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e86ea6-9d8d-4ec6-9ec7-f81f83850d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.746420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e86ea6-9d8d-4ec6-9ec7-f81f83850d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:17.747788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e86ea6-9d8d-4ec6-9ec7-f81f83850d25 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:17.761525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1084/2177] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:17.763540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e86ea6-9d8d-4ec6-9ec7-f81f83850d25 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:17.766986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06102e-ae59-4eb1-8baa-3570f895376a service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:17.772351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:17.773540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.773852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.774131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] PATH_INFO: `/v3/users/6cdd90b462394cf193cd4bb4886f2c50` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.774447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e86ea6-9d8d-4ec6-9ec7-f81f83850d25 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:17.775371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdd90b462394cf193cd4bb4886f2c50)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:17.777341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:17.782846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eca141-8225-496a-acbc-df7561762788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:17.782846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eca141-8225-496a-acbc-df7561762788 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.782846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eca141-8225-496a-acbc-df7561762788 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.782846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eca141-8225-496a-acbc-df7561762788 admin admin] PATH_INFO: `/v3/projects/837596c2e5d04f768a6becf23ae84288/users/9195a942dea8462fb1e78736fb5596be/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.797104 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eca141-8225-496a-acbc-df7561762788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=837596c2e5d04f768a6becf23ae84288, user_id=9195a942dea8462fb1e78736fb5596be, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:17.799986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eca141-8225-496a-acbc-df7561762788 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:17.832037 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:17.923842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1085/2178] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:17 2022] PUT /identity/v3/projects/837596c2e5d04f768a6becf23ae84288/users/9195a942dea8462fb1e78736fb5596be/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:17.934666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b2bed-453b-4659-b936-02561364e77a None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:17.934957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b2bed-453b-4659-b936-02561364e77a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:17.939039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b2bed-453b-4659-b936-02561364e77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:17.966710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-500b2bed-453b-4659-b936-02561364e77a None None] MFA Rules not processed for user `9195a942dea8462fb1e78736fb5596be`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:17.968067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-500b2bed-453b-4659-b936-02561364e77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:17.988222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdd90b462394cf193cd4bb4886f2c50'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:17.989408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdd90b462394cf193cd4bb4886f2c50'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:17.989989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdd90b462394cf193cd4bb4886f2c50'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:17.996935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdd90b462394cf193cd4bb4886f2c50'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:18.009750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdd90b462394cf193cd4bb4886f2c50'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:18.032853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1086/2179] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:18.033697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b2bed-453b-4659-b936-02561364e77a None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:18.034956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-500b2bed-453b-4659-b936-02561364e77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:18.090199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b2bed-453b-4659-b936-02561364e77a None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:18.094916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24df9d-2042-41f9-83d0-f93de07f3e23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:18.095581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24df9d-2042-41f9-83d0-f93de07f3e23 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:18.095858 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24df9d-2042-41f9-83d0-f93de07f3e23 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:18.096388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24df9d-2042-41f9-83d0-f93de07f3e23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:18.098692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-da24df9d-2042-41f9-83d0-f93de07f3e23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:18.100047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1094/2180] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:17 2022] DELETE /identity/v3/users/6cdd90b462394cf193cd4bb4886f2c50 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:18.100925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:18.111284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:18.168459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24df9d-2042-41f9-83d0-f93de07f3e23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:18.170164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24df9d-2042-41f9-83d0-f93de07f3e23 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:18.173253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d460f-7ff5-4dc9-8744-0c5b3439ba3e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:18.179291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d50def-4567-454b-95d7-fdb3e2ec1a87 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '142e6e44d6544f6a891ed3f913de74ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:18.182611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d50def-4567-454b-95d7-fdb3e2ec1a87 service glance] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:18.182922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d50def-4567-454b-95d7-fdb3e2ec1a87 service glance] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:18.183204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d50def-4567-454b-95d7-fdb3e2ec1a87 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:18.187626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1087/2181] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:18.276795 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d50def-4567-454b-95d7-fdb3e2ec1a87 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:18.281705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d50def-4567-454b-95d7-fdb3e2ec1a87 service glance] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:18.311108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1095/2182] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:18.472623 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:18.638006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24df9d-2042-41f9-83d0-f93de07f3e23 service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:18.638006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-da24df9d-2042-41f9-83d0-f93de07f3e23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:18.722537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24df9d-2042-41f9-83d0-f93de07f3e23 service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:18.784614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:18.786095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:18.786385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:18.786667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] PATH_INFO: `/v3/projects/20ee46ff0b834238b419fe98922a5d70` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:18.787894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20ee46ff0b834238b419fe98922a5d70)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:18.789979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:18.911918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20ee46ff0b834238b419fe98922a5d70', 'request_id': 'req-738f8e51-3972-4eb4-84e5-3ae6dc043f29'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:18.918220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20ee46ff0b834238b419fe98922a5d70', 'request_id': 'req-738f8e51-3972-4eb4-84e5-3ae6dc043f29'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:18.923606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20ee46ff0b834238b419fe98922a5d70', 'request_id': 'req-738f8e51-3972-4eb4-84e5-3ae6dc043f29'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:19.008074 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] The token cache is being invalidate because project 20ee46ff0b834238b419fe98922a5d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:19.008394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:19.009084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-738f8e51-3972-4eb4-84e5-3ae6dc043f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:19.010661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1088/2183] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:18 2022] DELETE /identity/v3/projects/20ee46ff0b834238b419fe98922a5d70 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:19.015676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c24ae6-d60b-423a-9cfc-e8db3439fbd8 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.016150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c24ae6-d60b-423a-9cfc-e8db3439fbd8 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.016498 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c24ae6-d60b-423a-9cfc-e8db3439fbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.037509 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e8c24ae6-d60b-423a-9cfc-e8db3439fbd8 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:19.038827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e8c24ae6-d60b-423a-9cfc-e8db3439fbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:19.097807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1096/2184] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:19.106393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67a4b2-5880-4975-b414-aaf9b3b2e0e2 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.106685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67a4b2-5880-4975-b414-aaf9b3b2e0e2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.106967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67a4b2-5880-4975-b414-aaf9b3b2e0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.125076 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6d67a4b2-5880-4975-b414-aaf9b3b2e0e2 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:19.126260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6d67a4b2-5880-4975-b414-aaf9b3b2e0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:19.144779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1089/2185] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:19.148099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c24ae6-d60b-423a-9cfc-e8db3439fbd8 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:19.159059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c24ae6-d60b-423a-9cfc-e8db3439fbd8 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:19.164725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18852d3-296b-42f2-8ded-b39bf7ad60bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:19.165395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18852d3-296b-42f2-8ded-b39bf7ad60bc admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.165686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18852d3-296b-42f2-8ded-b39bf7ad60bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.166036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18852d3-296b-42f2-8ded-b39bf7ad60bc admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.166809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18852d3-296b-42f2-8ded-b39bf7ad60bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:19.167122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18852d3-296b-42f2-8ded-b39bf7ad60bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:19.168846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18852d3-296b-42f2-8ded-b39bf7ad60bc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:19.180873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1097/2186] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:57:19.185417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67a4b2-5880-4975-b414-aaf9b3b2e0e2 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:19.194952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67a4b2-5880-4975-b414-aaf9b3b2e0e2 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:19.200506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38d93a-13ad-4779-8ca8-f5e14d709d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:19.202011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38d93a-13ad-4779-8ca8-f5e14d709d1a admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.202011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38d93a-13ad-4779-8ca8-f5e14d709d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.202011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38d93a-13ad-4779-8ca8-f5e14d709d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.202468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db38d93a-13ad-4779-8ca8-f5e14d709d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:19.204436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db38d93a-13ad-4779-8ca8-f5e14d709d1a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:19.307787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1090/2187] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:19 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:19.311969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18852d3-296b-42f2-8ded-b39bf7ad60bc admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:19.322531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18852d3-296b-42f2-8ded-b39bf7ad60bc admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:19.377283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38d93a-13ad-4779-8ca8-f5e14d709d1a admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:19.379853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-db38d93a-13ad-4779-8ca8-f5e14d709d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:19.403025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883679ba-f20f-4839-86c9-38c6b477df64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:19.403708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883679ba-f20f-4839-86c9-38c6b477df64 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.404053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883679ba-f20f-4839-86c9-38c6b477df64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.404260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883679ba-f20f-4839-86c9-38c6b477df64 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.404990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883679ba-f20f-4839-86c9-38c6b477df64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:19.407046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883679ba-f20f-4839-86c9-38c6b477df64 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:19.462145 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38d93a-13ad-4779-8ca8-f5e14d709d1a admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:19.467468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:19.469641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:19.470594 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.470866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.471150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] PATH_INFO: `/v3/users/14591cf9c9bb408395689602ad66785b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.472640 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=14591cf9c9bb408395689602ad66785b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:19.474803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:19.539481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1098/2188] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:19 2022] POST /identity/v3/users => generated 285 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:19.543627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883679ba-f20f-4839-86c9-38c6b477df64 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:19.553249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883679ba-f20f-4839-86c9-38c6b477df64 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:19.558024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd8fbb-51af-4443-b7ad-a48cf5d9acc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:19.558669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8fbb-51af-4443-b7ad-a48cf5d9acc3 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.558955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8fbb-51af-4443-b7ad-a48cf5d9acc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.559234 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8fbb-51af-4443-b7ad-a48cf5d9acc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.559976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd8fbb-51af-4443-b7ad-a48cf5d9acc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:19.561603 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd8fbb-51af-4443-b7ad-a48cf5d9acc3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:19.571749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1099/2189] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:19.575399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd8fbb-51af-4443-b7ad-a48cf5d9acc3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:19.585137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd8fbb-51af-4443-b7ad-a48cf5d9acc3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:19.589955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846668c-fe7b-4c7a-8286-1c56c82ff0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:19.590602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846668c-fe7b-4c7a-8286-1c56c82ff0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.591014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846668c-fe7b-4c7a-8286-1c56c82ff0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.591307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846668c-fe7b-4c7a-8286-1c56c82ff0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.592028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846668c-fe7b-4c7a-8286-1c56c82ff0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:19.593677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846668c-fe7b-4c7a-8286-1c56c82ff0a0 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:19.604327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1100/2190] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:19.608517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846668c-fe7b-4c7a-8286-1c56c82ff0a0 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:19.618915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846668c-fe7b-4c7a-8286-1c56c82ff0a0 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:19.623589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c379a-d432-418c-ab43-bf30f1932dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:19.624575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c379a-d432-418c-ab43-bf30f1932dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.624864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c379a-d432-418c-ab43-bf30f1932dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.625147 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c379a-d432-418c-ab43-bf30f1932dcf admin admin] PATH_INFO: `/v3/projects/85aedee3f4624ebca7a05c03afc26b90/users/f1f4967e10be436ba976b60a50185b20/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.640514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c379a-d432-418c-ab43-bf30f1932dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=85aedee3f4624ebca7a05c03afc26b90, user_id=f1f4967e10be436ba976b60a50185b20, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:19.643183 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c379a-d432-418c-ab43-bf30f1932dcf admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:19.680150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14591cf9c9bb408395689602ad66785b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:19.680870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14591cf9c9bb408395689602ad66785b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:19.693437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14591cf9c9bb408395689602ad66785b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:19.694101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14591cf9c9bb408395689602ad66785b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:19.701168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-50cdc2d7-a6ab-438e-8975-133b4e5294dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14591cf9c9bb408395689602ad66785b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:19.740746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1101/2191] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:19 2022] PUT /identity/v3/projects/85aedee3f4624ebca7a05c03afc26b90/users/f1f4967e10be436ba976b60a50185b20/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:19.750947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90408e9-259a-4207-8c64-2ed07b232fc5 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.751235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90408e9-259a-4207-8c64-2ed07b232fc5 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.751517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90408e9-259a-4207-8c64-2ed07b232fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.763290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1091/2192] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:19 2022] DELETE /identity/v3/users/14591cf9c9bb408395689602ad66785b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:19.772096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474bac7-f656-4114-ad10-55d24d2aa9a4 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.772504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474bac7-f656-4114-ad10-55d24d2aa9a4 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.772863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474bac7-f656-4114-ad10-55d24d2aa9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.779586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d90408e9-259a-4207-8c64-2ed07b232fc5 None None] MFA Rules not processed for user `f1f4967e10be436ba976b60a50185b20`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:19.780846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d90408e9-259a-4207-8c64-2ed07b232fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:19.790574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2474bac7-f656-4114-ad10-55d24d2aa9a4 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:19.791743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2474bac7-f656-4114-ad10-55d24d2aa9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:19.845119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1102/2193] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:19.856563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1092/2194] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:19.856701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90408e9-259a-4207-8c64-2ed07b232fc5 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:19.857996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d90408e9-259a-4207-8c64-2ed07b232fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:19.866515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474bac7-f656-4114-ad10-55d24d2aa9a4 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:19.867790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2474bac7-f656-4114-ad10-55d24d2aa9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:19.922048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90408e9-259a-4207-8c64-2ed07b232fc5 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:19.928688 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830dcaf-aa7d-4fb6-a070-93429dc68ae2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '142e6e44d6544f6a891ed3f913de74ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:19.929531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830dcaf-aa7d-4fb6-a070-93429dc68ae2 service glance] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.929824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830dcaf-aa7d-4fb6-a070-93429dc68ae2 service glance] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.930271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830dcaf-aa7d-4fb6-a070-93429dc68ae2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.943028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830dcaf-aa7d-4fb6-a070-93429dc68ae2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:19.944571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830dcaf-aa7d-4fb6-a070-93429dc68ae2 service glance] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:19.950845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474bac7-f656-4114-ad10-55d24d2aa9a4 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:19.954106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1103/2195] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:19 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:57:19.955021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb4171-a2f4-47b5-9696-f847b95b95da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:19.955741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb4171-a2f4-47b5-9696-f847b95b95da service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:19.956030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb4171-a2f4-47b5-9696-f847b95b95da service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:19.956306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb4171-a2f4-47b5-9696-f847b95b95da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:19.965994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb4171-a2f4-47b5-9696-f847b95b95da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:19.967407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb4171-a2f4-47b5-9696-f847b95b95da service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:19.977657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1093/2196] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:20.003992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:20.311529 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:20.427290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830dcaf-aa7d-4fb6-a070-93429dc68ae2 service glance] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:20.428854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6830dcaf-aa7d-4fb6-a070-93429dc68ae2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:20.444315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830dcaf-aa7d-4fb6-a070-93429dc68ae2 service glance] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:20.450026 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:20.451842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:20.452180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:20.452474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] PATH_INFO: `/v3/projects/536e030e42984f75bd593ce7bb77ce5b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:20.453798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=536e030e42984f75bd593ce7bb77ce5b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:20.455698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:20.587103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '536e030e42984f75bd593ce7bb77ce5b', 'request_id': 'req-e3924c5a-64fb-4406-9428-9c02c5fd0285'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:20.593803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '536e030e42984f75bd593ce7bb77ce5b', 'request_id': 'req-e3924c5a-64fb-4406-9428-9c02c5fd0285'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:20.599432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '536e030e42984f75bd593ce7bb77ce5b', 'request_id': 'req-e3924c5a-64fb-4406-9428-9c02c5fd0285'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:20.705432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] The token cache is being invalidate because project 536e030e42984f75bd593ce7bb77ce5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:20.705744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:20.706357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:20.707933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1104/2197] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:20 2022] DELETE /identity/v3/projects/536e030e42984f75bd593ce7bb77ce5b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:20.712059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb4171-a2f4-47b5-9696-f847b95b95da service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:20.721639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb4171-a2f4-47b5-9696-f847b95b95da service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:20.785023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:20.785023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:20.785023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:20.785023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] PATH_INFO: `/v3/users/172f88337a93407f8f3c4a8b05def02b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:20.785404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] RBAC: Authorizing `identity:delete_user(user_id=172f88337a93407f8f3c4a8b05def02b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:20.787479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:20.954483 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172f88337a93407f8f3c4a8b05def02b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:20.955300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172f88337a93407f8f3c4a8b05def02b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:20.967376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172f88337a93407f8f3c4a8b05def02b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:20.968060 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172f88337a93407f8f3c4a8b05def02b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:20.975563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c0b11e27-7e81-4fe2-931e-f71eae6b125e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172f88337a93407f8f3c4a8b05def02b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:21.049220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1094/2198] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:20 2022] DELETE /identity/v3/users/172f88337a93407f8f3c4a8b05def02b => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:21.295189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:21.353279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:21.462037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:21.463555 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:21.477680 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3924c5a-64fb-4406-9428-9c02c5fd0285 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:21.537805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:21.538703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:21.538983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:21.539261 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] PATH_INFO: `/v3/projects/3944d023a8ce495aa9213f9ef17b9e1a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:21.540485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3944d023a8ce495aa9213f9ef17b9e1a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:21.542427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:21.660181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3944d023a8ce495aa9213f9ef17b9e1a', 'request_id': 'req-89d74a90-1774-4339-97bd-3866c1668b15'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:21.666333 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3944d023a8ce495aa9213f9ef17b9e1a', 'request_id': 'req-89d74a90-1774-4339-97bd-3866c1668b15'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:21.672088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3944d023a8ce495aa9213f9ef17b9e1a', 'request_id': 'req-89d74a90-1774-4339-97bd-3866c1668b15'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:21.768202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] The token cache is being invalidate because project 3944d023a8ce495aa9213f9ef17b9e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:21.768527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:21.769141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-89d74a90-1774-4339-97bd-3866c1668b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:21.771381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1105/2199] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:21 2022] DELETE /identity/v3/projects/3944d023a8ce495aa9213f9ef17b9e1a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:21.776561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60015edf-991f-436c-861a-8b8a071b7a8e None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:21.777043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60015edf-991f-436c-861a-8b8a071b7a8e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:21.777191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60015edf-991f-436c-861a-8b8a071b7a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:21.795432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-60015edf-991f-436c-861a-8b8a071b7a8e None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:21.796748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-60015edf-991f-436c-861a-8b8a071b7a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:21.860398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1095/2200] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:21.868458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa6835-6703-48f0-a0b6-5d880660dcd7 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:21.869155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa6835-6703-48f0-a0b6-5d880660dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:21.869785 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa6835-6703-48f0-a0b6-5d880660dcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:21.891662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-89aa6835-6703-48f0-a0b6-5d880660dcd7 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:21.893314 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-89aa6835-6703-48f0-a0b6-5d880660dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:21.911358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1106/2201] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:21.912618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60015edf-991f-436c-861a-8b8a071b7a8e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:21.923448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60015edf-991f-436c-861a-8b8a071b7a8e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:21.931865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a05b0-ba1d-41de-a0d8-a648dae7d169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:21.933042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a05b0-ba1d-41de-a0d8-a648dae7d169 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:21.933472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a05b0-ba1d-41de-a0d8-a648dae7d169 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:21.933918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a05b0-ba1d-41de-a0d8-a648dae7d169 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:21.935020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a05b0-ba1d-41de-a0d8-a648dae7d169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:21.935473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a05b0-ba1d-41de-a0d8-a648dae7d169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:21.938009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a05b0-ba1d-41de-a0d8-a648dae7d169 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:21.954736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1096/2202] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:57:21.963683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa6835-6703-48f0-a0b6-5d880660dcd7 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:21.975321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa6835-6703-48f0-a0b6-5d880660dcd7 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:21.981561 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a130dc-bc27-4b13-8ddd-185dd49c2b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:21.982413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a130dc-bc27-4b13-8ddd-185dd49c2b42 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:21.982901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a130dc-bc27-4b13-8ddd-185dd49c2b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:21.983182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a130dc-bc27-4b13-8ddd-185dd49c2b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:21.983911 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a130dc-bc27-4b13-8ddd-185dd49c2b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:21.985876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a130dc-bc27-4b13-8ddd-185dd49c2b42 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:22.117816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1107/2203] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:21 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:22.121973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a05b0-ba1d-41de-a0d8-a648dae7d169 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:22.132682 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a05b0-ba1d-41de-a0d8-a648dae7d169 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:22.173256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:22.189735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982d2d7-b96f-48f1-8bfb-2ae6c70eb49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:22.190412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982d2d7-b96f-48f1-8bfb-2ae6c70eb49b admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:22.190697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982d2d7-b96f-48f1-8bfb-2ae6c70eb49b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:22.190976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982d2d7-b96f-48f1-8bfb-2ae6c70eb49b admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:22.191680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982d2d7-b96f-48f1-8bfb-2ae6c70eb49b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:22.194367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982d2d7-b96f-48f1-8bfb-2ae6c70eb49b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:22.221231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:22.300940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1097/2204] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:22 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:22.305654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a130dc-bc27-4b13-8ddd-185dd49c2b42 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:22.316607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a130dc-bc27-4b13-8ddd-185dd49c2b42 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:22.322101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd539b8b-8b50-44a6-b219-4cb2f349ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:22.322787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd539b8b-8b50-44a6-b219-4cb2f349ffe2 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:22.323070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd539b8b-8b50-44a6-b219-4cb2f349ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:22.324198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd539b8b-8b50-44a6-b219-4cb2f349ffe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:22.324198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd539b8b-8b50-44a6-b219-4cb2f349ffe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:22.325830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd539b8b-8b50-44a6-b219-4cb2f349ffe2 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:22.339189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1108/2205] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:22.343298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982d2d7-b96f-48f1-8bfb-2ae6c70eb49b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:22.354040 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982d2d7-b96f-48f1-8bfb-2ae6c70eb49b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:22.359119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb75682-1f49-471b-a061-0cd84b774dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:22.359721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb75682-1f49-471b-a061-0cd84b774dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:22.360011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb75682-1f49-471b-a061-0cd84b774dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:22.360289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb75682-1f49-471b-a061-0cd84b774dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:22.361130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb75682-1f49-471b-a061-0cd84b774dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:22.362703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb75682-1f49-471b-a061-0cd84b774dc8 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:22.373940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1098/2206] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:22.378007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd539b8b-8b50-44a6-b219-4cb2f349ffe2 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:22.388081 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd539b8b-8b50-44a6-b219-4cb2f349ffe2 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:22.393093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cda33-4c18-44f0-9021-7e850a522767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:22.393969 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cda33-4c18-44f0-9021-7e850a522767 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:22.394256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cda33-4c18-44f0-9021-7e850a522767 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:22.394539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cda33-4c18-44f0-9021-7e850a522767 admin admin] PATH_INFO: `/v3/projects/1d526febdb1d4993be787e25a8a78f7d/users/4b53d340dd6b4bba8d0b280407f94c8f/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:22.409511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cda33-4c18-44f0-9021-7e850a522767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d526febdb1d4993be787e25a8a78f7d, user_id=4b53d340dd6b4bba8d0b280407f94c8f, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:22.412747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cda33-4c18-44f0-9021-7e850a522767 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:22.512439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1109/2207] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:22 2022] PUT /identity/v3/projects/1d526febdb1d4993be787e25a8a78f7d/users/4b53d340dd6b4bba8d0b280407f94c8f/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:22.522746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c82e6-d53b-4739-bccf-362961fea66d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:22.523060 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c82e6-d53b-4739-bccf-362961fea66d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:22.523605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c82e6-d53b-4739-bccf-362961fea66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:22.550416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ca3c82e6-d53b-4739-bccf-362961fea66d None None] MFA Rules not processed for user `4b53d340dd6b4bba8d0b280407f94c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:22.552207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ca3c82e6-d53b-4739-bccf-362961fea66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:22.616513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1099/2208] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:22.629985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cda33-4c18-44f0-9021-7e850a522767 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:22.631819 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ad5cda33-4c18-44f0-9021-7e850a522767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:22.689319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cda33-4c18-44f0-9021-7e850a522767 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:22.693062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dfe72-fe6e-4395-b4da-35b22b134ca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:22.693710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dfe72-fe6e-4395-b4da-35b22b134ca7 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:22.693996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dfe72-fe6e-4395-b4da-35b22b134ca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:22.694274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dfe72-fe6e-4395-b4da-35b22b134ca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:22.702783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dfe72-fe6e-4395-b4da-35b22b134ca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:22.704221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dfe72-fe6e-4395-b4da-35b22b134ca7 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:22.713743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1110/2209] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:22.729523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:22.922094 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:23.866710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:24.070001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:24.340802 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:24.441879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:24.618965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:24.723533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:24.945936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:25.249264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:25.306896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:25.946235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:26.174897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:26.281540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:26.426153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:26.454704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:26.626013 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:26.654540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:26.780491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:26.808865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:26.917802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:26.986612 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:27.105009 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:27.197893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:27.704621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:27.838128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:27.984548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c82e6-d53b-4739-bccf-362961fea66d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:28.045335 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:28.049476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c82e6-d53b-4739-bccf-362961fea66d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:28.054670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:28.055566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:28.055848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:28.056122 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] PATH_INFO: `/v3/users/4b53d340dd6b4bba8d0b280407f94c8f` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:28.057350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b53d340dd6b4bba8d0b280407f94c8f)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:28.059270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:28.072701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:28.100947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:28.274936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b53d340dd6b4bba8d0b280407f94c8f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:28.275623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b53d340dd6b4bba8d0b280407f94c8f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:28.289211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b53d340dd6b4bba8d0b280407f94c8f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:28.290143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b53d340dd6b4bba8d0b280407f94c8f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:28.297771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b53d340dd6b4bba8d0b280407f94c8f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:28.341054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:28.376064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1100/2210] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:27 2022] DELETE /identity/v3/users/4b53d340dd6b4bba8d0b280407f94c8f => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:28.386909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da625d-9351-4902-90a6-b54272fb254b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:28.387321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da625d-9351-4902-90a6-b54272fb254b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:28.387860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da625d-9351-4902-90a6-b54272fb254b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:28.411823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e6da625d-9351-4902-90a6-b54272fb254b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:28.414274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e6da625d-9351-4902-90a6-b54272fb254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:28.496408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1111/2211] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:28.506095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:28.508618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:28.577947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b29ed7-5829-413b-93e2-9a5810c4ffd6 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:28.580968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:28.583473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc00ce-23d1-47f3-ad15-8b77defc4f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:28.584171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc00ce-23d1-47f3-ad15-8b77defc4f57 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:28.588890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc00ce-23d1-47f3-ad15-8b77defc4f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:28.588890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc00ce-23d1-47f3-ad15-8b77defc4f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:28.594769 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc00ce-23d1-47f3-ad15-8b77defc4f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:28.596251 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc00ce-23d1-47f3-ad15-8b77defc4f57 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:28.606590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1101/2212] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:28.701038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:29.157980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da625d-9351-4902-90a6-b54272fb254b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:29.159434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e6da625d-9351-4902-90a6-b54272fb254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:29.165261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:29.174798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da625d-9351-4902-90a6-b54272fb254b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:29.181396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:29.182422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:29.182751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:29.183150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] PATH_INFO: `/v3/projects/1d526febdb1d4993be787e25a8a78f7d` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:29.184679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d526febdb1d4993be787e25a8a78f7d)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:29.188747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:29.236146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:29.342317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d526febdb1d4993be787e25a8a78f7d', 'request_id': 'req-443330ca-d9d7-422e-beeb-03d2554b8cb1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:29.350038 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d526febdb1d4993be787e25a8a78f7d', 'request_id': 'req-443330ca-d9d7-422e-beeb-03d2554b8cb1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:29.356884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d526febdb1d4993be787e25a8a78f7d', 'request_id': 'req-443330ca-d9d7-422e-beeb-03d2554b8cb1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:29.436882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] The token cache is being invalidate because project 1d526febdb1d4993be787e25a8a78f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:29.437182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:29.437922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-443330ca-d9d7-422e-beeb-03d2554b8cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:29.439653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1112/2213] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:29 2022] DELETE /identity/v3/projects/1d526febdb1d4993be787e25a8a78f7d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:29.444575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5172060-01b9-4908-b796-4f4631a242f0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:29.445602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5172060-01b9-4908-b796-4f4631a242f0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:29.445788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5172060-01b9-4908-b796-4f4631a242f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:29.463985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b5172060-01b9-4908-b796-4f4631a242f0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:29.465224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b5172060-01b9-4908-b796-4f4631a242f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:29.527561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1102/2214] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:29.536322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0da527-2b3d-4864-b86e-b38966aca731 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:29.536648 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0da527-2b3d-4864-b86e-b38966aca731 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:29.536931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0da527-2b3d-4864-b86e-b38966aca731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:29.555624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-6a0da527-2b3d-4864-b86e-b38966aca731 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:29.556859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6a0da527-2b3d-4864-b86e-b38966aca731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:29.564814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1113/2215] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:29.569255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5172060-01b9-4908-b796-4f4631a242f0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:29.579310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5172060-01b9-4908-b796-4f4631a242f0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:29.583930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18372cbc-3a59-483e-91b2-a572b6548dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:29.584664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18372cbc-3a59-483e-91b2-a572b6548dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:29.584950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18372cbc-3a59-483e-91b2-a572b6548dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:29.585849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18372cbc-3a59-483e-91b2-a572b6548dc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:29.586583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18372cbc-3a59-483e-91b2-a572b6548dc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:29.586897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18372cbc-3a59-483e-91b2-a572b6548dc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:29.588509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18372cbc-3a59-483e-91b2-a572b6548dc4 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:29.601440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1103/2216] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:57:29.605728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0da527-2b3d-4864-b86e-b38966aca731 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:29.614917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0da527-2b3d-4864-b86e-b38966aca731 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:29.619409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:29.619606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83358c03-cc83-48b3-80be-ed4bc6474ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:29.620259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83358c03-cc83-48b3-80be-ed4bc6474ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:29.620577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83358c03-cc83-48b3-80be-ed4bc6474ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:29.620894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83358c03-cc83-48b3-80be-ed4bc6474ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:29.621546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83358c03-cc83-48b3-80be-ed4bc6474ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:29.623439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83358c03-cc83-48b3-80be-ed4bc6474ef3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:29.738201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1114/2217] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:29 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:29.745862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18372cbc-3a59-483e-91b2-a572b6548dc4 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:29.756748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18372cbc-3a59-483e-91b2-a572b6548dc4 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:29.787818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:29.816901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96effe8b-992b-40f4-93f4-0c0671082f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:29.817880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96effe8b-992b-40f4-93f4-0c0671082f50 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:29.818271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96effe8b-992b-40f4-93f4-0c0671082f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:29.818658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96effe8b-992b-40f4-93f4-0c0671082f50 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:29.819371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96effe8b-992b-40f4-93f4-0c0671082f50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:29.821942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96effe8b-992b-40f4-93f4-0c0671082f50 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:29.929678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:29.940222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1104/2218] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:29 2022] POST /identity/v3/users => generated 286 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:29.942458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83358c03-cc83-48b3-80be-ed4bc6474ef3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:29.954168 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83358c03-cc83-48b3-80be-ed4bc6474ef3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:29.959660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26842ce-2367-4afe-ab90-3487c912695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:29.961115 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26842ce-2367-4afe-ab90-3487c912695e admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:29.961399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26842ce-2367-4afe-ab90-3487c912695e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:29.961674 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26842ce-2367-4afe-ab90-3487c912695e admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:29.962461 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26842ce-2367-4afe-ab90-3487c912695e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:29.964288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26842ce-2367-4afe-ab90-3487c912695e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:29.982559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1115/2219] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:29.986263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96effe8b-992b-40f4-93f4-0c0671082f50 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:29.995717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:30.000905 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96effe8b-992b-40f4-93f4-0c0671082f50 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:30.006824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6147-1e0e-4adf-b792-007592dacd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:30.007704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6147-1e0e-4adf-b792-007592dacd77 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:30.008108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6147-1e0e-4adf-b792-007592dacd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:30.008635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f6147-1e0e-4adf-b792-007592dacd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:30.009496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f6147-1e0e-4adf-b792-007592dacd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:30.011142 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f6147-1e0e-4adf-b792-007592dacd77 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:30.024524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1105/2220] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:30.028844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26842ce-2367-4afe-ab90-3487c912695e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:30.039398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26842ce-2367-4afe-ab90-3487c912695e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:30.044751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940910b-5768-4db4-b01f-797c5a4686c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:30.045662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940910b-5768-4db4-b01f-797c5a4686c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:30.045936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940910b-5768-4db4-b01f-797c5a4686c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:30.046216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940910b-5768-4db4-b01f-797c5a4686c0 admin admin] PATH_INFO: `/v3/projects/67441f22b9c4455fa7975cca5a417185/users/c06753284d314b6b9ef28d77c05b452b/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:30.062959 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940910b-5768-4db4-b01f-797c5a4686c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67441f22b9c4455fa7975cca5a417185, user_id=c06753284d314b6b9ef28d77c05b452b, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:30.065790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940910b-5768-4db4-b01f-797c5a4686c0 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:30.100020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6147-1e0e-4adf-b792-007592dacd77 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:30.102691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d30f6147-1e0e-4adf-b792-007592dacd77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:30.115160 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f6147-1e0e-4adf-b792-007592dacd77 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:30.120255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:30.121149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:30.121427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:30.121707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] PATH_INFO: `/v3/users/7af1a1770a8340d1b8a9df25cee1afe5` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:30.122895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7af1a1770a8340d1b8a9df25cee1afe5)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:30.124839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:30.198384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1116/2221] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:30 2022] PUT /identity/v3/projects/67441f22b9c4455fa7975cca5a417185/users/c06753284d314b6b9ef28d77c05b452b/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:30.203226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9e707-f871-42e3-b1f3-e1008e99f31d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:30.203520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9e707-f871-42e3-b1f3-e1008e99f31d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:30.203802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9e707-f871-42e3-b1f3-e1008e99f31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:30.225042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d9b9e707-f871-42e3-b1f3-e1008e99f31d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:30.226887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d9b9e707-f871-42e3-b1f3-e1008e99f31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:30.287325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1117/2222] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:30.296870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9e707-f871-42e3-b1f3-e1008e99f31d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:30.298414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d9b9e707-f871-42e3-b1f3-e1008e99f31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:30.319160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:30.320204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7af1a1770a8340d1b8a9df25cee1afe5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:30.320944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7af1a1770a8340d1b8a9df25cee1afe5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:30.332956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7af1a1770a8340d1b8a9df25cee1afe5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:30.333645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7af1a1770a8340d1b8a9df25cee1afe5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:30.341987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7af1a1770a8340d1b8a9df25cee1afe5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:30.358274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9e707-f871-42e3-b1f3-e1008e99f31d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:30.362178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9edbe9-cabc-4d82-848e-7f1d3cf45e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:30.362835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9edbe9-cabc-4d82-848e-7f1d3cf45e99 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:30.363116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9edbe9-cabc-4d82-848e-7f1d3cf45e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:30.363401 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9edbe9-cabc-4d82-848e-7f1d3cf45e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:30.363683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:30.365240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-cd9edbe9-cabc-4d82-848e-7f1d3cf45e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:30.379124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9edbe9-cabc-4d82-848e-7f1d3cf45e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:30.380597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9edbe9-cabc-4d82-848e-7f1d3cf45e99 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:30.393465 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1118/2223] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:57:30.422390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1106/2224] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:30 2022] DELETE /identity/v3/users/7af1a1770a8340d1b8a9df25cee1afe5 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:30.428008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2443-f790-464e-8896-1025999a0932 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:30.428542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2443-f790-464e-8896-1025999a0932 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:30.429099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2443-f790-464e-8896-1025999a0932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:30.451004 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-82cb2443-f790-464e-8896-1025999a0932 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:30.452170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-82cb2443-f790-464e-8896-1025999a0932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:30.547631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1119/2225] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:30.558781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:30.560534 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:30.633021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae512acc-2efd-4a35-bf83-50c0c25ee841 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:30.637165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d50b9-51e4-4370-94cf-739aa34cbe3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:30.637964 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d50b9-51e4-4370-94cf-739aa34cbe3a service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:30.638250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d50b9-51e4-4370-94cf-739aa34cbe3a service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:30.638523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d50b9-51e4-4370-94cf-739aa34cbe3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:30.653215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d50b9-51e4-4370-94cf-739aa34cbe3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:30.654644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d50b9-51e4-4370-94cf-739aa34cbe3a service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:30.666475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1107/2226] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:57:30.711650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:30.957911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:31.182787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb2443-f790-464e-8896-1025999a0932 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:31.184414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-82cb2443-f790-464e-8896-1025999a0932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:31.198731 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb2443-f790-464e-8896-1025999a0932 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:31.204189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:31.205090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:31.205373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:31.205650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] PATH_INFO: `/v3/projects/168b2b67edb944d49fe4be84ec794d0a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:31.206860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=168b2b67edb944d49fe4be84ec794d0a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:31.210476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:31.338112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168b2b67edb944d49fe4be84ec794d0a', 'request_id': 'req-c4e68bb1-7a9b-4405-8928-534050d86f2b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:31.346457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168b2b67edb944d49fe4be84ec794d0a', 'request_id': 'req-c4e68bb1-7a9b-4405-8928-534050d86f2b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:31.349141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:31.354900 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168b2b67edb944d49fe4be84ec794d0a', 'request_id': 'req-c4e68bb1-7a9b-4405-8928-534050d86f2b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:31.447170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] The token cache is being invalidate because project 168b2b67edb944d49fe4be84ec794d0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:31.447460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:31.448130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c4e68bb1-7a9b-4405-8928-534050d86f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:31.449748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1120/2227] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:31 2022] DELETE /identity/v3/projects/168b2b67edb944d49fe4be84ec794d0a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:31.453863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d50b9-51e4-4370-94cf-739aa34cbe3a service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:31.463549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d50b9-51e4-4370-94cf-739aa34cbe3a service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:31.551126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:31.553367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:31.553874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:31.554635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] PATH_INFO: `/v3/users/f273762bf19042c68a11181894ec5b5c` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:31.556033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f273762bf19042c68a11181894ec5b5c)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:31.558158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:31.766036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f273762bf19042c68a11181894ec5b5c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:31.766684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f273762bf19042c68a11181894ec5b5c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:31.780798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f273762bf19042c68a11181894ec5b5c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:31.781493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f273762bf19042c68a11181894ec5b5c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:31.794298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f273762bf19042c68a11181894ec5b5c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:31.899950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1108/2228] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:31 2022] DELETE /identity/v3/users/f273762bf19042c68a11181894ec5b5c => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:31.900143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9aa74-493b-410c-a047-19af59f4bc3e None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:31.905381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9aa74-493b-410c-a047-19af59f4bc3e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:31.906034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9aa74-493b-410c-a047-19af59f4bc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:31.938312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-c2f9aa74-493b-410c-a047-19af59f4bc3e None None] MFA Rules not processed for user `c06753284d314b6b9ef28d77c05b452b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:31.939713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c2f9aa74-493b-410c-a047-19af59f4bc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:32.015625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1121/2229] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:32.033247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:32.034704 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:32.098500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb37980-8538-4eaf-b812-aca657dc1b4c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:32.105557 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c161f46-3321-4847-9b82-0981760d065d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:32.106376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c161f46-3321-4847-9b82-0981760d065d service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:32.106664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c161f46-3321-4847-9b82-0981760d065d service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:32.106944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c161f46-3321-4847-9b82-0981760d065d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:32.116083 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c161f46-3321-4847-9b82-0981760d065d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:32.118330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c161f46-3321-4847-9b82-0981760d065d service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:32.124570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:32.130401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1109/2230] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:32.436918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9aa74-493b-410c-a047-19af59f4bc3e None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:32.438377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c2f9aa74-493b-410c-a047-19af59f4bc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:32.568531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9aa74-493b-410c-a047-19af59f4bc3e None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:32.574014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:32.575212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:32.575501 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:32.575780 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] PATH_INFO: `/v3/projects/f9cea69a92c646f5966d5889275bbe6e` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:32.577034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9cea69a92c646f5966d5889275bbe6e)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:32.578945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:32.704922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9cea69a92c646f5966d5889275bbe6e', 'request_id': 'req-4835b9b6-6491-4745-98dd-4087196adc2c'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:32.711681 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9cea69a92c646f5966d5889275bbe6e', 'request_id': 'req-4835b9b6-6491-4745-98dd-4087196adc2c'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:32.717118 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9cea69a92c646f5966d5889275bbe6e', 'request_id': 'req-4835b9b6-6491-4745-98dd-4087196adc2c'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:32.779205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] The token cache is being invalidate because project f9cea69a92c646f5966d5889275bbe6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:32.779495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:32.780100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-4835b9b6-6491-4745-98dd-4087196adc2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:32.781663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1122/2231] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:32 2022] DELETE /identity/v3/projects/f9cea69a92c646f5966d5889275bbe6e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:32.786497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0629535-b498-49c0-87fc-42c8241dfc8c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:32.787021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0629535-b498-49c0-87fc-42c8241dfc8c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:32.787111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0629535-b498-49c0-87fc-42c8241dfc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:32.809841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b0629535-b498-49c0-87fc-42c8241dfc8c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:32.809841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b0629535-b498-49c0-87fc-42c8241dfc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:32.859367 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:32.868229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1110/2232] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:32.877117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16cdc0-f83a-430c-bcfc-82a8bf6c8181 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:32.877614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16cdc0-f83a-430c-bcfc-82a8bf6c8181 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:32.877899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16cdc0-f83a-430c-bcfc-82a8bf6c8181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:32.904540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ac16cdc0-f83a-430c-bcfc-82a8bf6c8181 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:32.904540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ac16cdc0-f83a-430c-bcfc-82a8bf6c8181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:32.920389 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1123/2233] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:32.922843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0629535-b498-49c0-87fc-42c8241dfc8c None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:32.932677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0629535-b498-49c0-87fc-42c8241dfc8c None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:32.937419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc549493-ee2f-4716-8b8c-5399ec15f738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:32.938084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc549493-ee2f-4716-8b8c-5399ec15f738 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:32.938368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc549493-ee2f-4716-8b8c-5399ec15f738 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:32.938648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc549493-ee2f-4716-8b8c-5399ec15f738 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:32.939462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc549493-ee2f-4716-8b8c-5399ec15f738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:32.939770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc549493-ee2f-4716-8b8c-5399ec15f738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:32.941411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc549493-ee2f-4716-8b8c-5399ec15f738 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:32.954233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1111/2234] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:57:32.958727 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16cdc0-f83a-430c-bcfc-82a8bf6c8181 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:32.968773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16cdc0-f83a-430c-bcfc-82a8bf6c8181 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:32.980741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5305ce5-d459-467f-b65f-554331cdbf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:32.981424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5305ce5-d459-467f-b65f-554331cdbf05 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:32.981707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5305ce5-d459-467f-b65f-554331cdbf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:32.985024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5305ce5-d459-467f-b65f-554331cdbf05 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:32.985024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5305ce5-d459-467f-b65f-554331cdbf05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:32.985024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5305ce5-d459-467f-b65f-554331cdbf05 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:33.079909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1124/2235] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:32 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:33.083902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc549493-ee2f-4716-8b8c-5399ec15f738 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:33.094039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc549493-ee2f-4716-8b8c-5399ec15f738 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:33.153009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a38011-4c90-46ef-9c54-ed1fdb80e053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:33.153691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a38011-4c90-46ef-9c54-ed1fdb80e053 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:33.153975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a38011-4c90-46ef-9c54-ed1fdb80e053 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:33.154406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a38011-4c90-46ef-9c54-ed1fdb80e053 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:33.154954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a38011-4c90-46ef-9c54-ed1fdb80e053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:33.156948 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a38011-4c90-46ef-9c54-ed1fdb80e053 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:33.160832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:33.238112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1112/2236] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:33 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:33.243055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5305ce5-d459-467f-b65f-554331cdbf05 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:33.253469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5305ce5-d459-467f-b65f-554331cdbf05 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:33.259139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d208cbe-81fc-4e95-9d10-32e194cc159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:33.259824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d208cbe-81fc-4e95-9d10-32e194cc159b admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:33.260109 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d208cbe-81fc-4e95-9d10-32e194cc159b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:33.260462 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d208cbe-81fc-4e95-9d10-32e194cc159b admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:33.261318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d208cbe-81fc-4e95-9d10-32e194cc159b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:33.262937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d208cbe-81fc-4e95-9d10-32e194cc159b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:33.274526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1125/2237] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:57:33.278567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a38011-4c90-46ef-9c54-ed1fdb80e053 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:33.290137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a38011-4c90-46ef-9c54-ed1fdb80e053 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:33.297638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffd1fa-e869-4550-b11e-3ecaa8d5aefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:33.298616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffd1fa-e869-4550-b11e-3ecaa8d5aefb admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:33.298918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffd1fa-e869-4550-b11e-3ecaa8d5aefb admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:33.299260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffd1fa-e869-4550-b11e-3ecaa8d5aefb admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:33.300008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffd1fa-e869-4550-b11e-3ecaa8d5aefb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:33.301652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ffd1fa-e869-4550-b11e-3ecaa8d5aefb admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:33.312863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1113/2238] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:33.317563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d208cbe-81fc-4e95-9d10-32e194cc159b admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:33.328138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d208cbe-81fc-4e95-9d10-32e194cc159b admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:33.332910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27dc4-c086-4a81-8cea-17f70efa111e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:33.333792 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27dc4-c086-4a81-8cea-17f70efa111e admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:33.334086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27dc4-c086-4a81-8cea-17f70efa111e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:33.334372 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27dc4-c086-4a81-8cea-17f70efa111e admin admin] PATH_INFO: `/v3/projects/49a60608e012487f9fe5239612caffbd/users/6101eee0c69e4f3d836df6f960a0c85f/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:33.349573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27dc4-c086-4a81-8cea-17f70efa111e admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a60608e012487f9fe5239612caffbd, user_id=6101eee0c69e4f3d836df6f960a0c85f, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:33.352425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27dc4-c086-4a81-8cea-17f70efa111e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:33.421093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1126/2239] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:33 2022] PUT /identity/v3/projects/49a60608e012487f9fe5239612caffbd/users/6101eee0c69e4f3d836df6f960a0c85f/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:33.432051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe47f7-cb14-4960-8f1a-22645bfcd425 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:33.432348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe47f7-cb14-4960-8f1a-22645bfcd425 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:33.432668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe47f7-cb14-4960-8f1a-22645bfcd425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:33.461186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-d9fe47f7-cb14-4960-8f1a-22645bfcd425 None None] MFA Rules not processed for user `6101eee0c69e4f3d836df6f960a0c85f`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:33.462405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d9fe47f7-cb14-4960-8f1a-22645bfcd425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:33.530289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1114/2240] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:33.534901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27dc4-c086-4a81-8cea-17f70efa111e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:33.602739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27dc4-c086-4a81-8cea-17f70efa111e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:33.608184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851fd5a-bd87-4245-afa4-de89bb37d4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:33.608885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851fd5a-bd87-4245-afa4-de89bb37d4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:33.609185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851fd5a-bd87-4245-afa4-de89bb37d4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:33.610328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851fd5a-bd87-4245-afa4-de89bb37d4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:33.611033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851fd5a-bd87-4245-afa4-de89bb37d4d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:33.614638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851fd5a-bd87-4245-afa4-de89bb37d4d0 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:33.662998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:33.724799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1127/2241] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:33 2022] POST /identity/v3/projects => generated 387 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:33.727157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe47f7-cb14-4960-8f1a-22645bfcd425 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:33.738259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe47f7-cb14-4960-8f1a-22645bfcd425 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:33.786508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:33.817343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780980f5-6bbc-4804-8800-80163ad5cc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:33.818034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780980f5-6bbc-4804-8800-80163ad5cc47 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:33.818318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780980f5-6bbc-4804-8800-80163ad5cc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:33.818592 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780980f5-6bbc-4804-8800-80163ad5cc47 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:33.819295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780980f5-6bbc-4804-8800-80163ad5cc47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:33.821365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780980f5-6bbc-4804-8800-80163ad5cc47 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:33.945204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1115/2242] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:33 2022] POST /identity/v3/users => generated 300 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:33.947257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851fd5a-bd87-4245-afa4-de89bb37d4d0 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:33.961591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851fd5a-bd87-4245-afa4-de89bb37d4d0 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:33.968806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa4514-f745-4952-89ce-14a03a0eb4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:33.969885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa4514-f745-4952-89ce-14a03a0eb4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:33.971180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa4514-f745-4952-89ce-14a03a0eb4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:33.971453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa4514-f745-4952-89ce-14a03a0eb4b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:33.972263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda31376-5009-48fa-a69c-4aa4c2a61c86 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:33.972586 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda31376-5009-48fa-a69c-4aa4c2a61c86 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:33.972802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa4514-f745-4952-89ce-14a03a0eb4b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:33.973218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda31376-5009-48fa-a69c-4aa4c2a61c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:33.974450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa4514-f745-4952-89ce-14a03a0eb4b1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:33.991050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1128/2243] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:33.992249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa4514-f745-4952-89ce-14a03a0eb4b1 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:33.992404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-eda31376-5009-48fa-a69c-4aa4c2a61c86 None None] MFA Rules not processed for user `c06753284d314b6b9ef28d77c05b452b`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:33.993584 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-eda31376-5009-48fa-a69c-4aa4c2a61c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:34.009865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa4514-f745-4952-89ce-14a03a0eb4b1 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:34.014807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc62f66-7fde-429a-a489-361b9a812442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:34.017436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc62f66-7fde-429a-a489-361b9a812442 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:34.017436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc62f66-7fde-429a-a489-361b9a812442 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:34.017436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc62f66-7fde-429a-a489-361b9a812442 admin admin] PATH_INFO: `/v3/projects/b1269a37099f49efb6c3eb182cc7bb8a/users/2531b7f9dc484bfc984d11dbc9bc1747/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:34.032967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc62f66-7fde-429a-a489-361b9a812442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1269a37099f49efb6c3eb182cc7bb8a, user_id=2531b7f9dc484bfc984d11dbc9bc1747, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:34.035612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc62f66-7fde-429a-a489-361b9a812442 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:34.037789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:34.068475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1116/2244] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:34.099334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:34.134329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1129/2245] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:33 2022] PUT /identity/v3/projects/b1269a37099f49efb6c3eb182cc7bb8a/users/2531b7f9dc484bfc984d11dbc9bc1747/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:34.138291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda31376-5009-48fa-a69c-4aa4c2a61c86 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:34.150074 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda31376-5009-48fa-a69c-4aa4c2a61c86 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:34.210207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a55a49-c440-4550-a222-b3310deda2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:34.211659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55a49-c440-4550-a222-b3310deda2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:34.211940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55a49-c440-4550-a222-b3310deda2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:34.212216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55a49-c440-4550-a222-b3310deda2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:34.212981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a55a49-c440-4550-a222-b3310deda2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:34.215240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a55a49-c440-4550-a222-b3310deda2a6 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:34.225743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1117/2246] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:34.230055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc62f66-7fde-429a-a489-361b9a812442 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:34.241006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc62f66-7fde-429a-a489-361b9a812442 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:34.246936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee7257-a772-4eed-b327-0b1a872342ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:34.248004 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee7257-a772-4eed-b327-0b1a872342ff admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:34.248351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee7257-a772-4eed-b327-0b1a872342ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:34.248713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee7257-a772-4eed-b327-0b1a872342ff admin admin] PATH_INFO: `/v3/projects/b1269a37099f49efb6c3eb182cc7bb8a/users/2531b7f9dc484bfc984d11dbc9bc1747/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:34.250940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee7257-a772-4eed-b327-0b1a872342ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1269a37099f49efb6c3eb182cc7bb8a, user_id=2531b7f9dc484bfc984d11dbc9bc1747, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:34.257929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee7257-a772-4eed-b327-0b1a872342ff admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:34.265160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1130/2247] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:34 2022] PUT /identity/v3/projects/b1269a37099f49efb6c3eb182cc7bb8a/users/2531b7f9dc484bfc984d11dbc9bc1747/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:34.276526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3168ead-cf9d-42c8-9bb3-528c4100183d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:34.276658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3168ead-cf9d-42c8-9bb3-528c4100183d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:34.276946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3168ead-cf9d-42c8-9bb3-528c4100183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:34.305480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a3168ead-cf9d-42c8-9bb3-528c4100183d None None] MFA Rules not processed for user `2531b7f9dc484bfc984d11dbc9bc1747`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:34.306688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a3168ead-cf9d-42c8-9bb3-528c4100183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:34.377235 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1118/2248] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:34.388612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee7257-a772-4eed-b327-0b1a872342ff admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:34.390474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7cee7257-a772-4eed-b327-0b1a872342ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:34.456571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee7257-a772-4eed-b327-0b1a872342ff admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:34.461204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0889cf-0d55-475d-94a1-d29f08264bd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:34.461833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0889cf-0d55-475d-94a1-d29f08264bd5 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:34.462145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0889cf-0d55-475d-94a1-d29f08264bd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:34.462427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0889cf-0d55-475d-94a1-d29f08264bd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:34.470893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:34.471605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0889cf-0d55-475d-94a1-d29f08264bd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:34.473101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0889cf-0d55-475d-94a1-d29f08264bd5 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:34.482539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1131/2249] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:34.552820 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:34.830709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:34.948258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:35.550710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:35.678046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:35.897077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3168ead-cf9d-42c8-9bb3-528c4100183d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:35.987105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3168ead-cf9d-42c8-9bb3-528c4100183d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:35.992578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:35.993774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:35.993774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:35.994108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] PATH_INFO: `/v3/users/6101eee0c69e4f3d836df6f960a0c85f` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:35.995366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6101eee0c69e4f3d836df6f960a0c85f)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:35.997336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:36.172837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6101eee0c69e4f3d836df6f960a0c85f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:36.173795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6101eee0c69e4f3d836df6f960a0c85f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:36.190123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6101eee0c69e4f3d836df6f960a0c85f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:36.190341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6101eee0c69e4f3d836df6f960a0c85f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:36.199559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6101eee0c69e4f3d836df6f960a0c85f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:36.261105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:36.272048 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1119/2250] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:35 2022] DELETE /identity/v3/users/6101eee0c69e4f3d836df6f960a0c85f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:57:36.277304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a22c-70ad-48c9-a3d2-b61c26b0683b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:36.277590 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a22c-70ad-48c9-a3d2-b61c26b0683b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:36.278036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a22c-70ad-48c9-a3d2-b61c26b0683b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:36.307423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b1b3a22c-70ad-48c9-a3d2-b61c26b0683b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:36.308686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b1b3a22c-70ad-48c9-a3d2-b61c26b0683b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:36.357613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:36.375828 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1132/2251] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:36.385305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:36.386752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:36.447668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37d50c-641b-4154-bfeb-ad6ebf8d0082 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:36.451908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dce953-0c04-4b73-9e6f-86018c0850c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:36.452631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dce953-0c04-4b73-9e6f-86018c0850c2 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:36.458360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dce953-0c04-4b73-9e6f-86018c0850c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:36.458360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dce953-0c04-4b73-9e6f-86018c0850c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:36.463894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dce953-0c04-4b73-9e6f-86018c0850c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:36.465445 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dce953-0c04-4b73-9e6f-86018c0850c2 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:36.475329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1120/2252] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:36.625377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:36.735552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:36.887529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3a22c-70ad-48c9-a3d2-b61c26b0683b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:36.889426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b1b3a22c-70ad-48c9-a3d2-b61c26b0683b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:36.903655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3a22c-70ad-48c9-a3d2-b61c26b0683b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:36.910776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:36.911467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:36.911779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:36.912031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] PATH_INFO: `/v3/projects/49a60608e012487f9fe5239612caffbd` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:36.913294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49a60608e012487f9fe5239612caffbd)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:36.915397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:36.967072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:37.073400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49a60608e012487f9fe5239612caffbd', 'request_id': 'req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:37.080331 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49a60608e012487f9fe5239612caffbd', 'request_id': 'req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:37.086021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49a60608e012487f9fe5239612caffbd', 'request_id': 'req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:37.175145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] The token cache is being invalidate because project 49a60608e012487f9fe5239612caffbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:37.175659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:37.176455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:37.178296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1133/2253] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:36 2022] DELETE /identity/v3/projects/49a60608e012487f9fe5239612caffbd => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:37.182858 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dce953-0c04-4b73-9e6f-86018c0850c2 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:37.197966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dce953-0c04-4b73-9e6f-86018c0850c2 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:37.259683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:37.260903 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:37.261638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:37.263290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:37.263723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] PATH_INFO: `/v3/users/2531b7f9dc484bfc984d11dbc9bc1747` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:37.265250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2531b7f9dc484bfc984d11dbc9bc1747)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:37.269708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:37.489531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:37.503163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2531b7f9dc484bfc984d11dbc9bc1747'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:37.503904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2531b7f9dc484bfc984d11dbc9bc1747'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:37.515986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2531b7f9dc484bfc984d11dbc9bc1747'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:37.516663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2531b7f9dc484bfc984d11dbc9bc1747'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:37.523413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b31ac2fa-de34-4616-abe3-856e01e9a1a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2531b7f9dc484bfc984d11dbc9bc1747'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:37.605614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1121/2254] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:37 2022] DELETE /identity/v3/users/2531b7f9dc484bfc984d11dbc9bc1747 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:37.867113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:38.114430 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:38.116017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:38.135652 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e9e8e-2495-4e2e-8346-cb7aee8fe513 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:38.157949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:38.195775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:38.197188 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:38.197476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:38.197755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] PATH_INFO: `/v3/projects/b1269a37099f49efb6c3eb182cc7bb8a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:38.198942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1269a37099f49efb6c3eb182cc7bb8a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:38.200997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:38.232075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:38.333201 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1269a37099f49efb6c3eb182cc7bb8a', 'request_id': 'req-2d2183bf-f5e3-427e-b1be-539bcb0174d0'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:38.349171 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1269a37099f49efb6c3eb182cc7bb8a', 'request_id': 'req-2d2183bf-f5e3-427e-b1be-539bcb0174d0'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:38.356234 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1269a37099f49efb6c3eb182cc7bb8a', 'request_id': 'req-2d2183bf-f5e3-427e-b1be-539bcb0174d0'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:38.426558 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:38.465634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] The token cache is being invalidate because project b1269a37099f49efb6c3eb182cc7bb8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:38.465928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:38.466559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-2d2183bf-f5e3-427e-b1be-539bcb0174d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:38.468108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1134/2255] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:38 2022] DELETE /identity/v3/projects/b1269a37099f49efb6c3eb182cc7bb8a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:38.473115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a001f7-bc6a-48a7-82d9-3a1e2ccf240c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:38.473617 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a001f7-bc6a-48a7-82d9-3a1e2ccf240c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:38.473902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a001f7-bc6a-48a7-82d9-3a1e2ccf240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:38.493006 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-82a001f7-bc6a-48a7-82d9-3a1e2ccf240c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:38.494908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-82a001f7-bc6a-48a7-82d9-3a1e2ccf240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:38.563128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1122/2256] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:38.572338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44a249-f2c8-4f97-81a4-f4dec2fccf95 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:38.572841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44a249-f2c8-4f97-81a4-f4dec2fccf95 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:38.573126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44a249-f2c8-4f97-81a4-f4dec2fccf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:38.594332 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-7a44a249-f2c8-4f97-81a4-f4dec2fccf95 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:38.600195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7a44a249-f2c8-4f97-81a4-f4dec2fccf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:38.604598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1135/2257] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:38.609045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a001f7-bc6a-48a7-82d9-3a1e2ccf240c None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:38.619969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a001f7-bc6a-48a7-82d9-3a1e2ccf240c None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:38.625533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead38bd4-3b88-4837-9266-0a0b7565de90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:38.626211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead38bd4-3b88-4837-9266-0a0b7565de90 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:38.626496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead38bd4-3b88-4837-9266-0a0b7565de90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:38.628446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead38bd4-3b88-4837-9266-0a0b7565de90 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:38.628446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead38bd4-3b88-4837-9266-0a0b7565de90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:38.628446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead38bd4-3b88-4837-9266-0a0b7565de90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:38.629430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead38bd4-3b88-4837-9266-0a0b7565de90 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:38.645513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1123/2258] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:57:38.647687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44a249-f2c8-4f97-81a4-f4dec2fccf95 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:38.660477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44a249-f2c8-4f97-81a4-f4dec2fccf95 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:38.665913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b2b06-ae89-403c-b877-bdaaf8cb3976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:38.666696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b2b06-ae89-403c-b877-bdaaf8cb3976 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:38.667101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b2b06-ae89-403c-b877-bdaaf8cb3976 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:38.667383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b2b06-ae89-403c-b877-bdaaf8cb3976 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:38.668112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b2b06-ae89-403c-b877-bdaaf8cb3976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:38.670349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b2b06-ae89-403c-b877-bdaaf8cb3976 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:38.761429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1136/2259] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:38 2022] POST /identity/v3/projects => generated 361 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:38.764437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead38bd4-3b88-4837-9266-0a0b7565de90 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:38.776500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead38bd4-3b88-4837-9266-0a0b7565de90 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:38.843595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5a824-e0be-4ccc-945f-d3ef35e2e73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:38.844395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5a824-e0be-4ccc-945f-d3ef35e2e73e admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:38.844787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5a824-e0be-4ccc-945f-d3ef35e2e73e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:38.845158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5a824-e0be-4ccc-945f-d3ef35e2e73e admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:38.845942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5a824-e0be-4ccc-945f-d3ef35e2e73e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:38.848173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5a824-e0be-4ccc-945f-d3ef35e2e73e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:38.918248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:38.932847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1124/2260] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:38 2022] POST /identity/v3/users => generated 281 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:38.934442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b2b06-ae89-403c-b877-bdaaf8cb3976 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:38.945491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b2b06-ae89-403c-b877-bdaaf8cb3976 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:38.950741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bd3b1-22f0-4832-9907-396d1561cf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:38.951577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd3b1-22f0-4832-9907-396d1561cf0e admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:38.951924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd3b1-22f0-4832-9907-396d1561cf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:38.952209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd3b1-22f0-4832-9907-396d1561cf0e admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:38.952979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bd3b1-22f0-4832-9907-396d1561cf0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:38.954583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bd3b1-22f0-4832-9907-396d1561cf0e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:38.968397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1137/2261] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:38.969815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5a824-e0be-4ccc-945f-d3ef35e2e73e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:38.981725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5a824-e0be-4ccc-945f-d3ef35e2e73e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:38.986792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9f8c0-9240-4bd8-9b5a-cd6569ec7ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:38.987468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9f8c0-9240-4bd8-9b5a-cd6569ec7ecb admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:38.987752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9f8c0-9240-4bd8-9b5a-cd6569ec7ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:38.988182 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9f8c0-9240-4bd8-9b5a-cd6569ec7ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:38.988987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9f8c0-9240-4bd8-9b5a-cd6569ec7ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:38.990718 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9f8c0-9240-4bd8-9b5a-cd6569ec7ecb admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:39.008275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1125/2262] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:39.011881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bd3b1-22f0-4832-9907-396d1561cf0e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:39.026214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bd3b1-22f0-4832-9907-396d1561cf0e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:39.031737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114d7d0-3a8e-44ec-87c7-4c22e97018ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:39.032735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114d7d0-3a8e-44ec-87c7-4c22e97018ee admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:39.033047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114d7d0-3a8e-44ec-87c7-4c22e97018ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:39.033294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114d7d0-3a8e-44ec-87c7-4c22e97018ee admin admin] PATH_INFO: `/v3/projects/045f56891bf94d61b3e67dc5fe1f950f/users/68b9b6828ef44e3ebb9ef730368c1b57/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:39.058635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114d7d0-3a8e-44ec-87c7-4c22e97018ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=045f56891bf94d61b3e67dc5fe1f950f, user_id=68b9b6828ef44e3ebb9ef730368c1b57, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:39.061404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114d7d0-3a8e-44ec-87c7-4c22e97018ee admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:39.147379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1138/2263] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:39 2022] PUT /identity/v3/projects/045f56891bf94d61b3e67dc5fe1f950f/users/68b9b6828ef44e3ebb9ef730368c1b57/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:39.159046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eec3e3-2fb8-493f-b5f9-73ab4404433a None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:39.159340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eec3e3-2fb8-493f-b5f9-73ab4404433a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:39.159625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eec3e3-2fb8-493f-b5f9-73ab4404433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:39.198846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-f2eec3e3-2fb8-493f-b5f9-73ab4404433a None None] MFA Rules not processed for user `68b9b6828ef44e3ebb9ef730368c1b57`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:39.200140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f2eec3e3-2fb8-493f-b5f9-73ab4404433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:39.272127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1126/2264] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:39.311441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114d7d0-3a8e-44ec-87c7-4c22e97018ee admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:39.313291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5114d7d0-3a8e-44ec-87c7-4c22e97018ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:39.377622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114d7d0-3a8e-44ec-87c7-4c22e97018ee admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:39.381827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd50076-861d-4419-83de-52fd4e65d51a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:39.382487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd50076-861d-4419-83de-52fd4e65d51a service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:39.382773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd50076-861d-4419-83de-52fd4e65d51a service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:39.383047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd50076-861d-4419-83de-52fd4e65d51a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:39.389346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:39.394172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd50076-861d-4419-83de-52fd4e65d51a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:39.395975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd50076-861d-4419-83de-52fd4e65d51a service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:39.409153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1139/2265] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:39.426802 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eec3e3-2fb8-493f-b5f9-73ab4404433a None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:39.509623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eec3e3-2fb8-493f-b5f9-73ab4404433a None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:39.514613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:39.515664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:39.515776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:39.516317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] PATH_INFO: `/v3/users/68b9b6828ef44e3ebb9ef730368c1b57` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:39.517490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b9b6828ef44e3ebb9ef730368c1b57)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:39.519699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:39.722580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b9b6828ef44e3ebb9ef730368c1b57'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:39.723261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b9b6828ef44e3ebb9ef730368c1b57'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:39.735377 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b9b6828ef44e3ebb9ef730368c1b57'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:39.736295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b9b6828ef44e3ebb9ef730368c1b57'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:39.743453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b9b6828ef44e3ebb9ef730368c1b57'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:39.747177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:39.838225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1127/2266] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:39 2022] DELETE /identity/v3/users/68b9b6828ef44e3ebb9ef730368c1b57 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:39.842929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1313a2-f71c-42c7-95ff-11061cf8eea0 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:39.843445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1313a2-f71c-42c7-95ff-11061cf8eea0 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:39.843709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1313a2-f71c-42c7-95ff-11061cf8eea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:39.863070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-bb1313a2-f71c-42c7-95ff-11061cf8eea0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:39.864581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bb1313a2-f71c-42c7-95ff-11061cf8eea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:39.929413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1140/2267] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:39.941167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:39.942637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:40.004923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a136a-4505-403a-988e-b59c558e90bc admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:40.008840 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627917c-e50f-436d-bc20-592597df8c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:40.010054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627917c-e50f-436d-bc20-592597df8c01 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:40.010341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627917c-e50f-436d-bc20-592597df8c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:40.010620 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627917c-e50f-436d-bc20-592597df8c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:40.020122 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627917c-e50f-436d-bc20-592597df8c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:40.021602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627917c-e50f-436d-bc20-592597df8c01 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:40.031685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1128/2268] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:40.460526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1313a2-f71c-42c7-95ff-11061cf8eea0 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:40.462075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bb1313a2-f71c-42c7-95ff-11061cf8eea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:40.476981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1313a2-f71c-42c7-95ff-11061cf8eea0 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:40.482379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:40.483253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:40.483534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:40.483809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] PATH_INFO: `/v3/projects/045f56891bf94d61b3e67dc5fe1f950f` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:40.485088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] RBAC: Authorizing `identity:delete_project(project_id=045f56891bf94d61b3e67dc5fe1f950f)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:40.487074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:40.586216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '045f56891bf94d61b3e67dc5fe1f950f', 'request_id': 'req-5adb0cfb-9804-4c8a-b655-58f11fe5943d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:40.594711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '045f56891bf94d61b3e67dc5fe1f950f', 'request_id': 'req-5adb0cfb-9804-4c8a-b655-58f11fe5943d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:40.603603 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '045f56891bf94d61b3e67dc5fe1f950f', 'request_id': 'req-5adb0cfb-9804-4c8a-b655-58f11fe5943d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:40.677437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] The token cache is being invalidate because project 045f56891bf94d61b3e67dc5fe1f950f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:57:40.677729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:40.678387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5adb0cfb-9804-4c8a-b655-58f11fe5943d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:40.679995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1141/2269] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:40 2022] DELETE /identity/v3/projects/045f56891bf94d61b3e67dc5fe1f950f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:40.685853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2241e-a879-4ad1-afc3-72f7833c0b21 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:40.686329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2241e-a879-4ad1-afc3-72f7833c0b21 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:40.686475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2241e-a879-4ad1-afc3-72f7833c0b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:40.704099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2ac2241e-a879-4ad1-afc3-72f7833c0b21 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:40.705392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2ac2241e-a879-4ad1-afc3-72f7833c0b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:40.775380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1129/2270] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:40.783102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a1a79-7bb5-47f8-8bd1-741f5d69a6bb None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:40.783395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a1a79-7bb5-47f8-8bd1-741f5d69a6bb None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:40.783698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a1a79-7bb5-47f8-8bd1-741f5d69a6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:40.802032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-403a1a79-7bb5-47f8-8bd1-741f5d69a6bb None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:40.803215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-403a1a79-7bb5-47f8-8bd1-741f5d69a6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:40.811133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1142/2271] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:40.816715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2241e-a879-4ad1-afc3-72f7833c0b21 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:40.826388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2241e-a879-4ad1-afc3-72f7833c0b21 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:40.831334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723065a-0bc9-4185-b6b5-385f08dc09d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:40.832091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723065a-0bc9-4185-b6b5-385f08dc09d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:40.832406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723065a-0bc9-4185-b6b5-385f08dc09d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:40.832695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723065a-0bc9-4185-b6b5-385f08dc09d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:40.833433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723065a-0bc9-4185-b6b5-385f08dc09d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:57:40.833745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723065a-0bc9-4185-b6b5-385f08dc09d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:40.835370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723065a-0bc9-4185-b6b5-385f08dc09d5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:40.847788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1130/2272] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:57:40.852512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a1a79-7bb5-47f8-8bd1-741f5d69a6bb None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:40.863507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a1a79-7bb5-47f8-8bd1-741f5d69a6bb None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:40.869351 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274894ce-01fa-4e0c-b16b-d54f1788c17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:40.870098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274894ce-01fa-4e0c-b16b-d54f1788c17e admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:40.870391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274894ce-01fa-4e0c-b16b-d54f1788c17e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:40.870980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274894ce-01fa-4e0c-b16b-d54f1788c17e admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:40.871739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274894ce-01fa-4e0c-b16b-d54f1788c17e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:40.873653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274894ce-01fa-4e0c-b16b-d54f1788c17e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:40.959672 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1143/2273] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:57:40 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:57:40.964289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723065a-0bc9-4185-b6b5-385f08dc09d5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:40.975288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723065a-0bc9-4185-b6b5-385f08dc09d5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:41.043288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1f968-9c1c-46fa-86f6-38b94c7606e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:41.043966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1f968-9c1c-46fa-86f6-38b94c7606e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:41.044252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1f968-9c1c-46fa-86f6-38b94c7606e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:41.044559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1f968-9c1c-46fa-86f6-38b94c7606e7 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:41.045289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1f968-9c1c-46fa-86f6-38b94c7606e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:41.047218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1f968-9c1c-46fa-86f6-38b94c7606e7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:41.139458 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1131/2274] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:57:40 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:57:41.143913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274894ce-01fa-4e0c-b16b-d54f1788c17e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:41.156950 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274894ce-01fa-4e0c-b16b-d54f1788c17e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:41.161933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57989924-32dc-4830-9736-12b018ea6bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:41.162666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57989924-32dc-4830-9736-12b018ea6bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:41.162949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57989924-32dc-4830-9736-12b018ea6bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:41.163249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57989924-32dc-4830-9736-12b018ea6bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:41.164228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57989924-32dc-4830-9736-12b018ea6bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:41.166415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57989924-32dc-4830-9736-12b018ea6bd6 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:41.177597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1144/2275] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:41.182282 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1f968-9c1c-46fa-86f6-38b94c7606e7 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:41.193796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1f968-9c1c-46fa-86f6-38b94c7606e7 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:41.199535 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81715f1-bf86-4a0e-a939-a92037b021dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:41.200221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81715f1-bf86-4a0e-a939-a92037b021dd admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:41.200564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81715f1-bf86-4a0e-a939-a92037b021dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:41.200815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81715f1-bf86-4a0e-a939-a92037b021dd admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:41.201561 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81715f1-bf86-4a0e-a939-a92037b021dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:41.203260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81715f1-bf86-4a0e-a939-a92037b021dd admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:41.215089 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1132/2276] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:57:41.219422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57989924-32dc-4830-9736-12b018ea6bd6 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:41.230298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57989924-32dc-4830-9736-12b018ea6bd6 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:41.235058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a4ae1-35e5-44df-a0d7-71235123b8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:41.235940 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a4ae1-35e5-44df-a0d7-71235123b8d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:41.236225 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a4ae1-35e5-44df-a0d7-71235123b8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:41.236532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a4ae1-35e5-44df-a0d7-71235123b8d5 admin admin] PATH_INFO: `/v3/projects/2a73a241a6ec473b8b16890fe997ba4e/users/8f8bc5cf16844078b77c6072205d380f/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:41.252747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a4ae1-35e5-44df-a0d7-71235123b8d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a73a241a6ec473b8b16890fe997ba4e, user_id=8f8bc5cf16844078b77c6072205d380f, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:41.255464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a4ae1-35e5-44df-a0d7-71235123b8d5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:41.318685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1145/2277] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:57:41 2022] PUT /identity/v3/projects/2a73a241a6ec473b8b16890fe997ba4e/users/8f8bc5cf16844078b77c6072205d380f/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:41.328587 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed82555-4387-45e7-a4e3-1bb93b41caa3 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:41.328883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed82555-4387-45e7-a4e3-1bb93b41caa3 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:41.329167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed82555-4387-45e7-a4e3-1bb93b41caa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:41.334077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:41.374714 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1ed82555-4387-45e7-a4e3-1bb93b41caa3 None None] MFA Rules not processed for user `8f8bc5cf16844078b77c6072205d380f`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:41.376698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1ed82555-4387-45e7-a4e3-1bb93b41caa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:41.464541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1133/2278] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:41.477748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a4ae1-35e5-44df-a0d7-71235123b8d5 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:41.480174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b49a4ae1-35e5-44df-a0d7-71235123b8d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:41.497392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:41.558625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a4ae1-35e5-44df-a0d7-71235123b8d5 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:41.562694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67593c4b-27ce-41dc-b7c0-9e1e549976d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:41.563228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67593c4b-27ce-41dc-b7c0-9e1e549976d3 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:41.563511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67593c4b-27ce-41dc-b7c0-9e1e549976d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:41.563791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67593c4b-27ce-41dc-b7c0-9e1e549976d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:41.572712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67593c4b-27ce-41dc-b7c0-9e1e549976d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:41.574181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67593c4b-27ce-41dc-b7c0-9e1e549976d3 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:41.583125 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1146/2279] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:57:43.448003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf11f7-5628-469f-930d-9d3445220183 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:43.448673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf11f7-5628-469f-930d-9d3445220183 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:43.448862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf11f7-5628-469f-930d-9d3445220183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:43.468549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8cbf11f7-5628-469f-930d-9d3445220183 None None] MFA Rules not processed for user `8f8bc5cf16844078b77c6072205d380f`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:43.469803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8cbf11f7-5628-469f-930d-9d3445220183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:43.477251 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1134/2280] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:57:51.855382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=29 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:53.462601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=45 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:58.721836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67593c4b-27ce-41dc-b7c0-9e1e549976d3 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:58.723936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-67593c4b-27ce-41dc-b7c0-9e1e549976d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:58.843104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67593c4b-27ce-41dc-b7c0-9e1e549976d3 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:58.847877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:58.848957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:58.849338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:58.849714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] PATH_INFO: `/v3/users/9195a942dea8462fb1e78736fb5596be` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:58.851169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9195a942dea8462fb1e78736fb5596be)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:58.853298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:59.083008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9195a942dea8462fb1e78736fb5596be'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:59.083746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9195a942dea8462fb1e78736fb5596be'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:59.084447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9195a942dea8462fb1e78736fb5596be'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:59.091655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9195a942dea8462fb1e78736fb5596be'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:59.103347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9195a942dea8462fb1e78736fb5596be'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:57:59.205658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1147/2281] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:57:58 2022] DELETE /identity/v3/users/9195a942dea8462fb1e78736fb5596be => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:57:59.210003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b80672-2e49-4dfb-b062-5208c9e7b27b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:59.210486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b80672-2e49-4dfb-b062-5208c9e7b27b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:59.210597 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b80672-2e49-4dfb-b062-5208c9e7b27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:59.239613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-f8b80672-2e49-4dfb-b062-5208c9e7b27b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:57:59.241397 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f8b80672-2e49-4dfb-b062-5208c9e7b27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:59.315237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1135/2282] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:57:59.328322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:59.330111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:59.396741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:57:59.427417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ebc9c-aca3-41d6-9819-698e4e4a2d10 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:59.431071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba792c43-3605-44b8-87f0-3e7794818a97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:59.431713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba792c43-3605-44b8-87f0-3e7794818a97 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:59.431995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba792c43-3605-44b8-87f0-3e7794818a97 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:59.432271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba792c43-3605-44b8-87f0-3e7794818a97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:59.441570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba792c43-3605-44b8-87f0-3e7794818a97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:59.443070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba792c43-3605-44b8-87f0-3e7794818a97 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:57:59.453240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1148/2283] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:57:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:57:59.929417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b80672-2e49-4dfb-b062-5208c9e7b27b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:57:59.933362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f8b80672-2e49-4dfb-b062-5208c9e7b27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:57:59.956809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b80672-2e49-4dfb-b062-5208c9e7b27b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:57:59.978822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:57:59.980185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:57:59.980503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:57:59.980782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] PATH_INFO: `/v3/projects/837596c2e5d04f768a6becf23ae84288` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:57:59.991466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=837596c2e5d04f768a6becf23ae84288)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:57:59.995934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:00.184613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '837596c2e5d04f768a6becf23ae84288', 'request_id': 'req-ac947418-e438-41f8-bf29-021e61e72f45'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:00.191760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '837596c2e5d04f768a6becf23ae84288', 'request_id': 'req-ac947418-e438-41f8-bf29-021e61e72f45'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:00.202503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '837596c2e5d04f768a6becf23ae84288', 'request_id': 'req-ac947418-e438-41f8-bf29-021e61e72f45'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:00.329988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] The token cache is being invalidate because project 837596c2e5d04f768a6becf23ae84288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:00.330646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:00.331132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ac947418-e438-41f8-bf29-021e61e72f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:00.333086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1136/2284] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:57:59 2022] DELETE /identity/v3/projects/837596c2e5d04f768a6becf23ae84288 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:00.338006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9ff4-d17c-4acd-a996-b8a557136db9 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:00.338437 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9ff4-d17c-4acd-a996-b8a557136db9 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:00.338573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9ff4-d17c-4acd-a996-b8a557136db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:00.389165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-53fc9ff4-d17c-4acd-a996-b8a557136db9 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:00.390393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-53fc9ff4-d17c-4acd-a996-b8a557136db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:00.458578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1149/2285] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:00.466950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1bd4d-0fa3-4c50-b747-e381576565af None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:00.467241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1bd4d-0fa3-4c50-b747-e381576565af None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:00.467519 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1bd4d-0fa3-4c50-b747-e381576565af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:00.486498 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-0cc1bd4d-0fa3-4c50-b747-e381576565af None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:00.487709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0cc1bd4d-0fa3-4c50-b747-e381576565af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:00.496988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1137/2286] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:00.501348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc9ff4-d17c-4acd-a996-b8a557136db9 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:00.511624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc9ff4-d17c-4acd-a996-b8a557136db9 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:00.516611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a137409-9763-4fce-9077-53240678c52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:00.517283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a137409-9763-4fce-9077-53240678c52c admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:00.517557 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a137409-9763-4fce-9077-53240678c52c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:00.517833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a137409-9763-4fce-9077-53240678c52c admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:00.518558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a137409-9763-4fce-9077-53240678c52c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:58:00.518864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a137409-9763-4fce-9077-53240678c52c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:00.520460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a137409-9763-4fce-9077-53240678c52c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:00.538071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1150/2287] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:58:00.542336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1bd4d-0fa3-4c50-b747-e381576565af None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:00.552457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1bd4d-0fa3-4c50-b747-e381576565af None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:00.557277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ff35a-7803-4d4c-8eae-76ba6c9aab2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:00.558234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ff35a-7803-4d4c-8eae-76ba6c9aab2b admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:00.571709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ff35a-7803-4d4c-8eae-76ba6c9aab2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:00.571709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ff35a-7803-4d4c-8eae-76ba6c9aab2b admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:00.571709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ff35a-7803-4d4c-8eae-76ba6c9aab2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:00.571709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ff35a-7803-4d4c-8eae-76ba6c9aab2b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:00.656306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1138/2288] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:58:00 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:00.660687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a137409-9763-4fce-9077-53240678c52c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:00.670785 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a137409-9763-4fce-9077-53240678c52c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:00.731326 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3f4b8-ac3b-4a94-b3dc-1954159773e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:00.732268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3f4b8-ac3b-4a94-b3dc-1954159773e1 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:00.732268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3f4b8-ac3b-4a94-b3dc-1954159773e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:00.732568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3f4b8-ac3b-4a94-b3dc-1954159773e1 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:00.733265 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3f4b8-ac3b-4a94-b3dc-1954159773e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:00.735165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3f4b8-ac3b-4a94-b3dc-1954159773e1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:00.810066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1151/2289] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:58:00 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:00.814155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ff35a-7803-4d4c-8eae-76ba6c9aab2b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:00.824225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ff35a-7803-4d4c-8eae-76ba6c9aab2b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:00.828866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8c031-5c34-4df8-8c96-9f501b6cdb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:00.829526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8c031-5c34-4df8-8c96-9f501b6cdb98 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:00.829813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8c031-5c34-4df8-8c96-9f501b6cdb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:00.830094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8c031-5c34-4df8-8c96-9f501b6cdb98 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:00.830819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f8c031-5c34-4df8-8c96-9f501b6cdb98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:00.832429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f8c031-5c34-4df8-8c96-9f501b6cdb98 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:00.844461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1139/2290] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:00.848320 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3f4b8-ac3b-4a94-b3dc-1954159773e1 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:00.859212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3f4b8-ac3b-4a94-b3dc-1954159773e1 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:00.863805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7d74d-efb3-41a6-a647-f8ee2741ac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:00.864508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7d74d-efb3-41a6-a647-f8ee2741ac4d admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:00.868454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7d74d-efb3-41a6-a647-f8ee2741ac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:00.868454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7d74d-efb3-41a6-a647-f8ee2741ac4d admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:00.868454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7d74d-efb3-41a6-a647-f8ee2741ac4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:00.868454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7d74d-efb3-41a6-a647-f8ee2741ac4d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:00.877589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1152/2291] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:00.881708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8c031-5c34-4df8-8c96-9f501b6cdb98 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:00.895522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8c031-5c34-4df8-8c96-9f501b6cdb98 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:00.905673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f39f06-bf57-4f1d-9c8e-d92dff64f566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:00.906581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f39f06-bf57-4f1d-9c8e-d92dff64f566 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:00.906982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f39f06-bf57-4f1d-9c8e-d92dff64f566 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:00.907143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f39f06-bf57-4f1d-9c8e-d92dff64f566 admin admin] PATH_INFO: `/v3/projects/804b869705bb4aa8a6fae5015ccbc6d1/users/6aba7c99a4ed4b348f7d5dd87ca3b208/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:00.924708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f39f06-bf57-4f1d-9c8e-d92dff64f566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=804b869705bb4aa8a6fae5015ccbc6d1, user_id=6aba7c99a4ed4b348f7d5dd87ca3b208, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:00.927365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f39f06-bf57-4f1d-9c8e-d92dff64f566 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:01.010640 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1140/2292] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:00 2022] PUT /identity/v3/projects/804b869705bb4aa8a6fae5015ccbc6d1/users/6aba7c99a4ed4b348f7d5dd87ca3b208/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:01.020883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1dab5-dfd4-43c3-8a2d-7fbe0f085abc None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:01.022226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1dab5-dfd4-43c3-8a2d-7fbe0f085abc None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:01.022226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1dab5-dfd4-43c3-8a2d-7fbe0f085abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:01.040527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:01.050154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-1bb1dab5-dfd4-43c3-8a2d-7fbe0f085abc None None] MFA Rules not processed for user `6aba7c99a4ed4b348f7d5dd87ca3b208`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:01.051367 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1bb1dab5-dfd4-43c3-8a2d-7fbe0f085abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:01.115302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1153/2293] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:01.129524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f39f06-bf57-4f1d-9c8e-d92dff64f566 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:01.131323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-34f39f06-bf57-4f1d-9c8e-d92dff64f566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:01.146210 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:01.188534 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f39f06-bf57-4f1d-9c8e-d92dff64f566 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:01.192219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c94f5-8424-4e41-b892-0f9f07c0ec6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:01.192897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c94f5-8424-4e41-b892-0f9f07c0ec6a service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:01.193184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c94f5-8424-4e41-b892-0f9f07c0ec6a service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:01.193461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c94f5-8424-4e41-b892-0f9f07c0ec6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:01.205221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c94f5-8424-4e41-b892-0f9f07c0ec6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:01.207079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c94f5-8424-4e41-b892-0f9f07c0ec6a service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:01.219050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1141/2294] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:11.199098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=28 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:11.432440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=20 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:20.059485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1dab5-dfd4-43c3-8a2d-7fbe0f085abc None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:20.061678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1bb1dab5-dfd4-43c3-8a2d-7fbe0f085abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:20.121452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c94f5-8424-4e41-b892-0f9f07c0ec6a service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:20.146436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1dab5-dfd4-43c3-8a2d-7fbe0f085abc None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:20.152383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:20.154919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:20.155251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:20.155549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] PATH_INFO: `/v3/users/08a091a4ac1546d6b390089a5bc45da7` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:20.156944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a091a4ac1546d6b390089a5bc45da7)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:20.158910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:20.192720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c94f5-8424-4e41-b892-0f9f07c0ec6a service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:20.197420 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:20.198300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:20.198588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:20.198898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] PATH_INFO: `/v3/users/6aba7c99a4ed4b348f7d5dd87ca3b208` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:20.200104 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aba7c99a4ed4b348f7d5dd87ca3b208)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:20.202103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:20.373578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a091a4ac1546d6b390089a5bc45da7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:20.374235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a091a4ac1546d6b390089a5bc45da7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:20.374848 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a091a4ac1546d6b390089a5bc45da7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:20.381834 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a091a4ac1546d6b390089a5bc45da7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:20.394596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-5f5f9b95-4772-489a-af0d-aa91ef35f8d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a091a4ac1546d6b390089a5bc45da7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:20.441345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aba7c99a4ed4b348f7d5dd87ca3b208'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:20.441998 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aba7c99a4ed4b348f7d5dd87ca3b208'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:20.454056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aba7c99a4ed4b348f7d5dd87ca3b208'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:20.454692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aba7c99a4ed4b348f7d5dd87ca3b208'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:20.461821 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-fbc34845-9bf8-4086-a7f4-feb6cc385464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aba7c99a4ed4b348f7d5dd87ca3b208'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:20.501907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1154/2295] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:20 2022] DELETE /identity/v3/users/08a091a4ac1546d6b390089a5bc45da7 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:20.503523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a137d-66c2-4efb-8418-cca7f51e57dd None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:20.503850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a137d-66c2-4efb-8418-cca7f51e57dd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:20.504165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a137d-66c2-4efb-8418-cca7f51e57dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:20.529439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-029a137d-66c2-4efb-8418-cca7f51e57dd None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:20.529439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-029a137d-66c2-4efb-8418-cca7f51e57dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:20.587548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1142/2296] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:20 2022] DELETE /identity/v3/users/6aba7c99a4ed4b348f7d5dd87ca3b208 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:20.602909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f737a-1b10-45cc-a7cd-d604d6e5b2f7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:20.603316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f737a-1b10-45cc-a7cd-d604d6e5b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:20.603720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f737a-1b10-45cc-a7cd-d604d6e5b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:20.617935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1155/2297] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:20.619136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a137d-66c2-4efb-8418-cca7f51e57dd None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:20.620650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-029a137d-66c2-4efb-8418-cca7f51e57dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:20.629525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ca5f737a-1b10-45cc-a7cd-d604d6e5b2f7 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:20.631104 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ca5f737a-1b10-45cc-a7cd-d604d6e5b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:20.655813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:20.703923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a137d-66c2-4efb-8418-cca7f51e57dd None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:20.730152 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1143/2298] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:20.740562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f737a-1b10-45cc-a7cd-d604d6e5b2f7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:20.761680 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:20.766879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f76649-5daa-41a5-9112-fa40cfa61b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:20.767707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f76649-5daa-41a5-9112-fa40cfa61b02 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:20.767997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f76649-5daa-41a5-9112-fa40cfa61b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:20.768277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f76649-5daa-41a5-9112-fa40cfa61b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:20.779004 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f76649-5daa-41a5-9112-fa40cfa61b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:20.780500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f76649-5daa-41a5-9112-fa40cfa61b02 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:20.790089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1156/2299] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:20.801875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f737a-1b10-45cc-a7cd-d604d6e5b2f7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:20.805423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01622a68-04d8-4e22-a529-5dc821b2f7e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:20.806081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01622a68-04d8-4e22-a529-5dc821b2f7e8 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:20.806367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01622a68-04d8-4e22-a529-5dc821b2f7e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:20.806647 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01622a68-04d8-4e22-a529-5dc821b2f7e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:20.815848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01622a68-04d8-4e22-a529-5dc821b2f7e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:20.817303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01622a68-04d8-4e22-a529-5dc821b2f7e8 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:20.827383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1144/2300] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:58:21.407362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f76649-5daa-41a5-9112-fa40cfa61b02 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:21.410371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-23f76649-5daa-41a5-9112-fa40cfa61b02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:21.426668 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f76649-5daa-41a5-9112-fa40cfa61b02 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:21.434467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:21.436034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:21.436387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:21.436763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] PATH_INFO: `/v3/projects/19a6858b332d4c7cb12a9ae7cb7963b3` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:21.439147 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a6858b332d4c7cb12a9ae7cb7963b3)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:21.441500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:21.452683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01622a68-04d8-4e22-a529-5dc821b2f7e8 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:21.455466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-01622a68-04d8-4e22-a529-5dc821b2f7e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:21.483118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01622a68-04d8-4e22-a529-5dc821b2f7e8 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:21.493770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:21.494791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:21.495344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:21.495785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] PATH_INFO: `/v3/projects/804b869705bb4aa8a6fae5015ccbc6d1` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:21.497813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] RBAC: Authorizing `identity:delete_project(project_id=804b869705bb4aa8a6fae5015ccbc6d1)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:21.499728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:21.610903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a6858b332d4c7cb12a9ae7cb7963b3', 'request_id': 'req-ba23a60f-0879-4763-aa0d-91b873c7dc04'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:21.617006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a6858b332d4c7cb12a9ae7cb7963b3', 'request_id': 'req-ba23a60f-0879-4763-aa0d-91b873c7dc04'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:21.645468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a6858b332d4c7cb12a9ae7cb7963b3', 'request_id': 'req-ba23a60f-0879-4763-aa0d-91b873c7dc04'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:21.645570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804b869705bb4aa8a6fae5015ccbc6d1', 'request_id': 'req-592b6666-691a-4117-bb4b-b68c8a58c81e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:21.649930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804b869705bb4aa8a6fae5015ccbc6d1', 'request_id': 'req-592b6666-691a-4117-bb4b-b68c8a58c81e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:21.655910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804b869705bb4aa8a6fae5015ccbc6d1', 'request_id': 'req-592b6666-691a-4117-bb4b-b68c8a58c81e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:21.755858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] The token cache is being invalidate because project 19a6858b332d4c7cb12a9ae7cb7963b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:21.756153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:21.756840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:21.758805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1157/2301] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:21 2022] DELETE /identity/v3/projects/19a6858b332d4c7cb12a9ae7cb7963b3 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:21.759590 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:21.760933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:21.832025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba23a60f-0879-4763-aa0d-91b873c7dc04 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:21.837294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:21.842813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] The token cache is being invalidate because project 804b869705bb4aa8a6fae5015ccbc6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:21.843109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:21.843997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-592b6666-691a-4117-bb4b-b68c8a58c81e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:21.845812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1145/2302] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:21 2022] DELETE /identity/v3/projects/804b869705bb4aa8a6fae5015ccbc6d1 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:21.850333 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4f126-aca7-4936-b8d4-e58337c04910 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:21.850333 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4f126-aca7-4936-b8d4-e58337c04910 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:21.850333 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4f126-aca7-4936-b8d4-e58337c04910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:21.872994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-92e4f126-aca7-4936-b8d4-e58337c04910 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:21.874271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-92e4f126-aca7-4936-b8d4-e58337c04910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:21.901832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:21.903630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:21.904022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:21.904521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] PATH_INFO: `/v3/users/63b379593c2743999e59c7e23ee27af1` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:21.905996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b379593c2743999e59c7e23ee27af1)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:21.908577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:21.949263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1146/2303] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:21.958135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68cfd3-0c66-45d6-862e-dc3c1eeb3d23 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:21.958540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68cfd3-0c66-45d6-862e-dc3c1eeb3d23 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:21.958924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68cfd3-0c66-45d6-862e-dc3c1eeb3d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:21.991139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-fd68cfd3-0c66-45d6-862e-dc3c1eeb3d23 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:21.992388 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-fd68cfd3-0c66-45d6-862e-dc3c1eeb3d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:22.007347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1147/2304] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:22.008944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2305b-bebb-4ac5-8291-aad927c2cb02 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.009421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2305b-bebb-4ac5-8291-aad927c2cb02 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.009860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2305b-bebb-4ac5-8291-aad927c2cb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.037743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a5a2305b-bebb-4ac5-8291-aad927c2cb02 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:22.039203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a5a2305b-bebb-4ac5-8291-aad927c2cb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:22.061075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1148/2305] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:22.061075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c087bcd-8b1b-4f67-8232-6cbb5ad446b2 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.061075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c087bcd-8b1b-4f67-8232-6cbb5ad446b2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.061075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c087bcd-8b1b-4f67-8232-6cbb5ad446b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.078557 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b379593c2743999e59c7e23ee27af1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:22.079279 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1c087bcd-8b1b-4f67-8232-6cbb5ad446b2 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:22.079480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b379593c2743999e59c7e23ee27af1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:22.080382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b379593c2743999e59c7e23ee27af1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:22.081889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1c087bcd-8b1b-4f67-8232-6cbb5ad446b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:22.088734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b379593c2743999e59c7e23ee27af1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:22.089176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1149/2306] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:22.089980 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c087bcd-8b1b-4f67-8232-6cbb5ad446b2 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.091809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1c087bcd-8b1b-4f67-8232-6cbb5ad446b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:22.103469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c087bcd-8b1b-4f67-8232-6cbb5ad446b2 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.104162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b379593c2743999e59c7e23ee27af1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:22.108250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e8a37-1042-4841-b2da-22a14c16d32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.108935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e8a37-1042-4841-b2da-22a14c16d32f admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.109219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e8a37-1042-4841-b2da-22a14c16d32f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.109494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e8a37-1042-4841-b2da-22a14c16d32f admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.110221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e8a37-1042-4841-b2da-22a14c16d32f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:58:22.110530 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e8a37-1042-4841-b2da-22a14c16d32f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.112186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e8a37-1042-4841-b2da-22a14c16d32f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.127472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1150/2307] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:58:22.127472 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e8a37-1042-4841-b2da-22a14c16d32f admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.136190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e8a37-1042-4841-b2da-22a14c16d32f admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.142499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26386932-2218-4373-bcdf-b87e269ff86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.143192 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26386932-2218-4373-bcdf-b87e269ff86e admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.143479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26386932-2218-4373-bcdf-b87e269ff86e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.143758 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26386932-2218-4373-bcdf-b87e269ff86e admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.144526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26386932-2218-4373-bcdf-b87e269ff86e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:58:22.144837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26386932-2218-4373-bcdf-b87e269ff86e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.146424 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26386932-2218-4373-bcdf-b87e269ff86e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.157459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:22.169861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1151/2308] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:58:22.173764 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26386932-2218-4373-bcdf-b87e269ff86e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.184386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26386932-2218-4373-bcdf-b87e269ff86e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.193025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1158/2309] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:21 2022] DELETE /identity/v3/users/63b379593c2743999e59c7e23ee27af1 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:22.194394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5271bab-ddae-49e8-9b6f-158dfa6cd7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.195807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5271bab-ddae-49e8-9b6f-158dfa6cd7fc admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.196297 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5271bab-ddae-49e8-9b6f-158dfa6cd7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.196692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5271bab-ddae-49e8-9b6f-158dfa6cd7fc admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.197403 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5271bab-ddae-49e8-9b6f-158dfa6cd7fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.199216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.199593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5271bab-ddae-49e8-9b6f-158dfa6cd7fc admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.274511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ca278-455d-446f-9f63-4e747a2d6452 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.284241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040eb1b-d82b-4089-9d97-ba857d5f8b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.284757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040eb1b-d82b-4089-9d97-ba857d5f8b92 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.286030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040eb1b-d82b-4089-9d97-ba857d5f8b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.286030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040eb1b-d82b-4089-9d97-ba857d5f8b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.286532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040eb1b-d82b-4089-9d97-ba857d5f8b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.291311 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040eb1b-d82b-4089-9d97-ba857d5f8b92 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.308005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1152/2310] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:58:22 2022] POST /identity/v3/projects => generated 361 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:22.309585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32133a43-e90e-42aa-b74c-cdd59ecc2903 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.310007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32133a43-e90e-42aa-b74c-cdd59ecc2903 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.310414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32133a43-e90e-42aa-b74c-cdd59ecc2903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.330529 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-32133a43-e90e-42aa-b74c-cdd59ecc2903 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:22.335645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-32133a43-e90e-42aa-b74c-cdd59ecc2903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:22.403380 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1159/2311] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:58:22 2022] POST /identity/v3/projects => generated 383 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:22.404696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040eb1b-d82b-4089-9d97-ba857d5f8b92 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.420608 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1153/2312] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:22.421692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32133a43-e90e-42aa-b74c-cdd59ecc2903 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.424904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:22.434912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32133a43-e90e-42aa-b74c-cdd59ecc2903 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.519162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040eb1b-d82b-4089-9d97-ba857d5f8b92 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.527499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46366bd-4b55-4c93-8f82-80d7feba03ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.528867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46366bd-4b55-4c93-8f82-80d7feba03ee admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.529502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46366bd-4b55-4c93-8f82-80d7feba03ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.530068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46366bd-4b55-4c93-8f82-80d7feba03ee admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.531024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46366bd-4b55-4c93-8f82-80d7feba03ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.533800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46366bd-4b55-4c93-8f82-80d7feba03ee admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.549982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd804de-9b5a-4cf0-9419-002117ae7022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.551327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd804de-9b5a-4cf0-9419-002117ae7022 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.551764 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd804de-9b5a-4cf0-9419-002117ae7022 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.555018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd804de-9b5a-4cf0-9419-002117ae7022 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.555018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd804de-9b5a-4cf0-9419-002117ae7022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.555748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd804de-9b5a-4cf0-9419-002117ae7022 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.655038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1154/2313] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:58:22 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:22.656103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd804de-9b5a-4cf0-9419-002117ae7022 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.659111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1160/2314] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:58:22 2022] POST /identity/v3/users => generated 281 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:22.662023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-4dd804de-9b5a-4cf0-9419-002117ae7022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:22.665656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46366bd-4b55-4c93-8f82-80d7feba03ee admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.676795 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46366bd-4b55-4c93-8f82-80d7feba03ee admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.682562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25aed5f-5dff-4f11-b9ad-1741957a4c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.683312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25aed5f-5dff-4f11-b9ad-1741957a4c38 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.683687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25aed5f-5dff-4f11-b9ad-1741957a4c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.684726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25aed5f-5dff-4f11-b9ad-1741957a4c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.685654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25aed5f-5dff-4f11-b9ad-1741957a4c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.687451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25aed5f-5dff-4f11-b9ad-1741957a4c38 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.709341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1161/2315] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:22.710391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25aed5f-5dff-4f11-b9ad-1741957a4c38 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.724441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25aed5f-5dff-4f11-b9ad-1741957a4c38 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.731180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd804de-9b5a-4cf0-9419-002117ae7022 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.733556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9ed6d-ba6f-4997-a9ef-9482bf066fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.734246 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9ed6d-ba6f-4997-a9ef-9482bf066fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.734529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9ed6d-ba6f-4997-a9ef-9482bf066fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.735282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9ed6d-ba6f-4997-a9ef-9482bf066fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.735650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9ed6d-ba6f-4997-a9ef-9482bf066fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.737328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9ed6d-ba6f-4997-a9ef-9482bf066fb5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.739878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754d4d2-abe2-4b67-8ecf-9be13b43b026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.740591 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754d4d2-abe2-4b67-8ecf-9be13b43b026 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.740882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754d4d2-abe2-4b67-8ecf-9be13b43b026 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.741165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754d4d2-abe2-4b67-8ecf-9be13b43b026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.749718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1162/2316] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:22.750376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9ed6d-ba6f-4997-a9ef-9482bf066fb5 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.756058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754d4d2-abe2-4b67-8ecf-9be13b43b026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.757673 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0754d4d2-abe2-4b67-8ecf-9be13b43b026 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.760676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9ed6d-ba6f-4997-a9ef-9482bf066fb5 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.767299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323fa89-d784-4bb5-b397-d84d2eeb1c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.768415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323fa89-d784-4bb5-b397-d84d2eeb1c03 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.768415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323fa89-d784-4bb5-b397-d84d2eeb1c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.768593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323fa89-d784-4bb5-b397-d84d2eeb1c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.769984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323fa89-d784-4bb5-b397-d84d2eeb1c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.771221 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323fa89-d784-4bb5-b397-d84d2eeb1c03 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.781250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1155/2317] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:22.785576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754d4d2-abe2-4b67-8ecf-9be13b43b026 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.785703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1163/2318] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:22.786860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323fa89-d784-4bb5-b397-d84d2eeb1c03 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.797632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754d4d2-abe2-4b67-8ecf-9be13b43b026 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.798228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323fa89-d784-4bb5-b397-d84d2eeb1c03 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.804303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286252ca-1d7b-4631-8e30-ef54fc034b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.804778 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2204a3c-6265-4cad-9bf8-28f7f0c6448d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.805934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286252ca-1d7b-4631-8e30-ef54fc034b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.806229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2204a3c-6265-4cad-9bf8-28f7f0c6448d admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.806542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286252ca-1d7b-4631-8e30-ef54fc034b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.806671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2204a3c-6265-4cad-9bf8-28f7f0c6448d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.807089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286252ca-1d7b-4631-8e30-ef54fc034b51 admin admin] PATH_INFO: `/v3/projects/bd07b008154d412a856b5a381680b5a9/users/13035fc79c124492930fca0a48db6551/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.808019 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2204a3c-6265-4cad-9bf8-28f7f0c6448d admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.811665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2204a3c-6265-4cad-9bf8-28f7f0c6448d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.817200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2204a3c-6265-4cad-9bf8-28f7f0c6448d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.822723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:22.827938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286252ca-1d7b-4631-8e30-ef54fc034b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd07b008154d412a856b5a381680b5a9, user_id=13035fc79c124492930fca0a48db6551, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.834160 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1156/2319] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:22.834307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286252ca-1d7b-4631-8e30-ef54fc034b51 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.836337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2204a3c-6265-4cad-9bf8-28f7f0c6448d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:22.850158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2204a3c-6265-4cad-9bf8-28f7f0c6448d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:22.857952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c098be-8fae-4f20-a240-40713190b981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:22.858997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c098be-8fae-4f20-a240-40713190b981 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.859349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c098be-8fae-4f20-a240-40713190b981 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.859703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c098be-8fae-4f20-a240-40713190b981 admin admin] PATH_INFO: `/v3/projects/5df2f448827740aa84665e079f469100/users/14411a9a02c5492b8878dedd1e3e775d/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.878017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c098be-8fae-4f20-a240-40713190b981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df2f448827740aa84665e079f469100, user_id=14411a9a02c5492b8878dedd1e3e775d, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:22.881187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c098be-8fae-4f20-a240-40713190b981 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:22.937273 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1164/2320] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:22 2022] PUT /identity/v3/projects/bd07b008154d412a856b5a381680b5a9/users/13035fc79c124492930fca0a48db6551/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:22.947739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4ef5c-b0c8-4e35-ac61-61f1cadc2f2d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:22.948090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4ef5c-b0c8-4e35-ac61-61f1cadc2f2d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:22.948512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4ef5c-b0c8-4e35-ac61-61f1cadc2f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:22.978851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a0d4ef5c-b0c8-4e35-ac61-61f1cadc2f2d None None] MFA Rules not processed for user `13035fc79c124492930fca0a48db6551`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:22.980182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a0d4ef5c-b0c8-4e35-ac61-61f1cadc2f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:22.990298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1157/2321] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:22 2022] PUT /identity/v3/projects/5df2f448827740aa84665e079f469100/users/14411a9a02c5492b8878dedd1e3e775d/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:23.001978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41f22f-33f3-46a3-b96d-a805f7bf435a None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:23.001978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41f22f-33f3-46a3-b96d-a805f7bf435a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:23.001978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41f22f-33f3-46a3-b96d-a805f7bf435a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:23.037208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-db41f22f-33f3-46a3-b96d-a805f7bf435a None None] MFA Rules not processed for user `14411a9a02c5492b8878dedd1e3e775d`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:23.038666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-db41f22f-33f3-46a3-b96d-a805f7bf435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:23.126835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1165/2322] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:23.138799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4ef5c-b0c8-4e35-ac61-61f1cadc2f2d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:23.140610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a0d4ef5c-b0c8-4e35-ac61-61f1cadc2f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:23.175830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1158/2323] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:23.184313 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41f22f-33f3-46a3-b96d-a805f7bf435a None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:23.185770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-db41f22f-33f3-46a3-b96d-a805f7bf435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:23.218073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:23.276084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4ef5c-b0c8-4e35-ac61-61f1cadc2f2d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:23.281458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05619bf-eb61-4312-ab69-df4becb69f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:23.282872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05619bf-eb61-4312-ab69-df4becb69f64 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:23.285405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41f22f-33f3-46a3-b96d-a805f7bf435a None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:23.286036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05619bf-eb61-4312-ab69-df4becb69f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:23.286810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05619bf-eb61-4312-ab69-df4becb69f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:23.290342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c095b1-0dda-456c-a5bc-a98da7d3cc62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:23.293061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c095b1-0dda-456c-a5bc-a98da7d3cc62 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:23.293061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c095b1-0dda-456c-a5bc-a98da7d3cc62 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:23.293061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c095b1-0dda-456c-a5bc-a98da7d3cc62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:23.303927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c095b1-0dda-456c-a5bc-a98da7d3cc62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:23.306148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c095b1-0dda-456c-a5bc-a98da7d3cc62 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:23.308249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05619bf-eb61-4312-ab69-df4becb69f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:23.315952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05619bf-eb61-4312-ab69-df4becb69f64 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:23.320322 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1159/2324] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:58:23.346764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1166/2325] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:58:23.424582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c095b1-0dda-456c-a5bc-a98da7d3cc62 service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:23.426410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-57c095b1-0dda-456c-a5bc-a98da7d3cc62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:23.560835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c095b1-0dda-456c-a5bc-a98da7d3cc62 service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:23.566759 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:23.567642 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:23.567927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:23.568202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] PATH_INFO: `/v3/projects/98a5e68ec7ca4f88b875a558beef15c9` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:23.569443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98a5e68ec7ca4f88b875a558beef15c9)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:23.571459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:23.573169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:23.697690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98a5e68ec7ca4f88b875a558beef15c9', 'request_id': 'req-88da1124-976f-481d-96d0-b42f5a793110'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:23.704149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98a5e68ec7ca4f88b875a558beef15c9', 'request_id': 'req-88da1124-976f-481d-96d0-b42f5a793110'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:23.710588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98a5e68ec7ca4f88b875a558beef15c9', 'request_id': 'req-88da1124-976f-481d-96d0-b42f5a793110'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:23.777781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] The token cache is being invalidate because project 98a5e68ec7ca4f88b875a558beef15c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:23.778216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:23.778927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:23.780548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1160/2326] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:23 2022] DELETE /identity/v3/projects/98a5e68ec7ca4f88b875a558beef15c9 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:23.784474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05619bf-eb61-4312-ab69-df4becb69f64 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:23.795917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05619bf-eb61-4312-ab69-df4becb69f64 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:23.854788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:23.855673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:23.856634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:23.856824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] PATH_INFO: `/v3/users/30abcfea60a94267aa3f5c3f0a14ed5a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:23.858032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30abcfea60a94267aa3f5c3f0a14ed5a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:23.859984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:24.183162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30abcfea60a94267aa3f5c3f0a14ed5a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:24.184020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30abcfea60a94267aa3f5c3f0a14ed5a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:24.184780 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30abcfea60a94267aa3f5c3f0a14ed5a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:24.195267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30abcfea60a94267aa3f5c3f0a14ed5a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:24.210105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc6be3bd-2857-4481-8e7c-56c38b4eda60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30abcfea60a94267aa3f5c3f0a14ed5a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:24.311783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1167/2327] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:23 2022] DELETE /identity/v3/users/30abcfea60a94267aa3f5c3f0a14ed5a => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:24.831745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:24.833374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:24.846757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da1124-976f-481d-96d0-b42f5a793110 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:24.916941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:24.918600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:24.919010 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:24.919451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] PATH_INFO: `/v3/projects/fc7364dd0f2a4ed68b083c9d5e6b1bd1` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:24.921478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7364dd0f2a4ed68b083c9d5e6b1bd1)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:24.924845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:25.011021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d947b-2694-49e6-bbc1-17ca5d98d08b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:25.011532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d947b-2694-49e6-bbc1-17ca5d98d08b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:25.011698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d947b-2694-49e6-bbc1-17ca5d98d08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:25.030787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-463d947b-2694-49e6-bbc1-17ca5d98d08b None None] MFA Rules not processed for user `c06753284d314b6b9ef28d77c05b452b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:25.032008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-463d947b-2694-49e6-bbc1-17ca5d98d08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:25.068511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:25.071650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7364dd0f2a4ed68b083c9d5e6b1bd1', 'request_id': 'req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:25.078067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7364dd0f2a4ed68b083c9d5e6b1bd1', 'request_id': 'req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:25.083676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7364dd0f2a4ed68b083c9d5e6b1bd1', 'request_id': 'req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:25.092722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1168/2328] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:25.174249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] The token cache is being invalidate because project fc7364dd0f2a4ed68b083c9d5e6b1bd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:25.174249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:25.174249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b2dd2a94-adc3-45c8-a6a3-0b4c2641a155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:25.174249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1161/2329] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:24 2022] DELETE /identity/v3/projects/fc7364dd0f2a4ed68b083c9d5e6b1bd1 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:25.181865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11555e3-9a4b-4b54-b01a-bcbaee1d18c5 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:25.182158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11555e3-9a4b-4b54-b01a-bcbaee1d18c5 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:25.182436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11555e3-9a4b-4b54-b01a-bcbaee1d18c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:25.202257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b11555e3-9a4b-4b54-b01a-bcbaee1d18c5 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:25.203768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b11555e3-9a4b-4b54-b01a-bcbaee1d18c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:25.282478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1169/2330] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:25.295358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7327a8-3019-4fd3-98ab-5276346f1972 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:25.295656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7327a8-3019-4fd3-98ab-5276346f1972 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:25.295941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7327a8-3019-4fd3-98ab-5276346f1972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:25.318409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1c7327a8-3019-4fd3-98ab-5276346f1972 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:25.319933 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1c7327a8-3019-4fd3-98ab-5276346f1972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:25.331354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1162/2331] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:25.336096 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11555e3-9a4b-4b54-b01a-bcbaee1d18c5 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:25.350566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11555e3-9a4b-4b54-b01a-bcbaee1d18c5 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:25.355312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c84429-c523-4fd1-9018-e2f7c557edbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:25.356007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c84429-c523-4fd1-9018-e2f7c557edbf admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:25.356297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c84429-c523-4fd1-9018-e2f7c557edbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:25.356597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c84429-c523-4fd1-9018-e2f7c557edbf admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:25.357481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c84429-c523-4fd1-9018-e2f7c557edbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:58:25.357797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c84429-c523-4fd1-9018-e2f7c557edbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:25.359407 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c84429-c523-4fd1-9018-e2f7c557edbf admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:25.374778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1170/2332] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:58:25.377684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7327a8-3019-4fd3-98ab-5276346f1972 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:25.393081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7327a8-3019-4fd3-98ab-5276346f1972 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:25.398908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb9395-63e5-44ef-af49-adba39996af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:25.399654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb9395-63e5-44ef-af49-adba39996af2 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:25.399942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb9395-63e5-44ef-af49-adba39996af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:25.400222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb9395-63e5-44ef-af49-adba39996af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:25.404851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb9395-63e5-44ef-af49-adba39996af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:25.404851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb9395-63e5-44ef-af49-adba39996af2 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:25.431628 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:25.506415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1163/2333] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:58:25 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:25.510717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c84429-c523-4fd1-9018-e2f7c557edbf admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:25.522664 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c84429-c523-4fd1-9018-e2f7c557edbf admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:25.589330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3d943-0f99-4f6b-a68e-c30c20e7ebb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:25.589330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3d943-0f99-4f6b-a68e-c30c20e7ebb0 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:25.589751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3d943-0f99-4f6b-a68e-c30c20e7ebb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:25.589914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3d943-0f99-4f6b-a68e-c30c20e7ebb0 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:25.591517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3d943-0f99-4f6b-a68e-c30c20e7ebb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:25.594104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3d943-0f99-4f6b-a68e-c30c20e7ebb0 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:25.713701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1171/2334] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:58:25 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:25.717798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb9395-63e5-44ef-af49-adba39996af2 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:25.729321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb9395-63e5-44ef-af49-adba39996af2 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:25.734610 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb1a24-3107-43fe-9bcb-0bd0f20f3b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:25.735361 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb1a24-3107-43fe-9bcb-0bd0f20f3b73 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:25.735655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb1a24-3107-43fe-9bcb-0bd0f20f3b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:25.736084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb1a24-3107-43fe-9bcb-0bd0f20f3b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:25.736877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb1a24-3107-43fe-9bcb-0bd0f20f3b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:25.738522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb1a24-3107-43fe-9bcb-0bd0f20f3b73 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:25.751832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1164/2335] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:25.755860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3d943-0f99-4f6b-a68e-c30c20e7ebb0 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:25.767081 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3d943-0f99-4f6b-a68e-c30c20e7ebb0 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:25.774261 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936541a2-c5e5-43db-b781-59a2217de6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:25.774973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936541a2-c5e5-43db-b781-59a2217de6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:25.775461 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936541a2-c5e5-43db-b781-59a2217de6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:25.776181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936541a2-c5e5-43db-b781-59a2217de6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:25.780607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936541a2-c5e5-43db-b781-59a2217de6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:25.780607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936541a2-c5e5-43db-b781-59a2217de6a8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:25.795337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1172/2336] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:25.796774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb1a24-3107-43fe-9bcb-0bd0f20f3b73 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:25.808055 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb1a24-3107-43fe-9bcb-0bd0f20f3b73 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:25.813676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362beee7-db2f-400d-b1df-fdc98f976514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:25.814611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362beee7-db2f-400d-b1df-fdc98f976514 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:25.814895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362beee7-db2f-400d-b1df-fdc98f976514 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:25.815170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362beee7-db2f-400d-b1df-fdc98f976514 admin admin] PATH_INFO: `/v3/projects/a807c9a5c0f540509dd8ed3e2ac1506d/users/3aa2ed15b110472c84989304e86947af/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:25.833736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362beee7-db2f-400d-b1df-fdc98f976514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a807c9a5c0f540509dd8ed3e2ac1506d, user_id=3aa2ed15b110472c84989304e86947af, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:25.836935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362beee7-db2f-400d-b1df-fdc98f976514 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:25.946224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1165/2337] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:25 2022] PUT /identity/v3/projects/a807c9a5c0f540509dd8ed3e2ac1506d/users/3aa2ed15b110472c84989304e86947af/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:25.958492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804bd5a-c727-40bc-8e5c-7209e5027701 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:25.958791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804bd5a-c727-40bc-8e5c-7209e5027701 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:25.960124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804bd5a-c727-40bc-8e5c-7209e5027701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:26.003121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-3804bd5a-c727-40bc-8e5c-7209e5027701 None None] MFA Rules not processed for user `3aa2ed15b110472c84989304e86947af`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:26.004400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3804bd5a-c727-40bc-8e5c-7209e5027701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:26.061374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:26.088819 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad1d63-7e2e-4551-b512-1dd87fb1f09e None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:26.089132 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad1d63-7e2e-4551-b512-1dd87fb1f09e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:26.089367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad1d63-7e2e-4551-b512-1dd87fb1f09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:26.130812 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b1ad1d63-7e2e-4551-b512-1dd87fb1f09e None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:26.132799 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b1ad1d63-7e2e-4551-b512-1dd87fb1f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:26.140764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1173/2338] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:26.148621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804bd5a-c727-40bc-8e5c-7209e5027701 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:26.221372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1166/2339] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 15:58:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:26.250423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804bd5a-c727-40bc-8e5c-7209e5027701 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:26.258113 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0f19-cd62-4f4e-bd07-976829d5c345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:26.258967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0f19-cd62-4f4e-bd07-976829d5c345 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:26.259396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0f19-cd62-4f4e-bd07-976829d5c345 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:26.259815 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0f19-cd62-4f4e-bd07-976829d5c345 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:26.260722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d0f19-cd62-4f4e-bd07-976829d5c345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:26.262956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d0f19-cd62-4f4e-bd07-976829d5c345 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:26.284114 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad1d63-7e2e-4551-b512-1dd87fb1f09e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:26.286945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b1ad1d63-7e2e-4551-b512-1dd87fb1f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:26.388814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad1d63-7e2e-4551-b512-1dd87fb1f09e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:26.394382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65d360-db81-47df-a902-c9989d3fdee3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:26.395156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65d360-db81-47df-a902-c9989d3fdee3 service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:26.395439 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65d360-db81-47df-a902-c9989d3fdee3 service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:26.395716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65d360-db81-47df-a902-c9989d3fdee3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:26.405298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65d360-db81-47df-a902-c9989d3fdee3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:26.406896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65d360-db81-47df-a902-c9989d3fdee3 service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:26.418631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1167/2340] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 15:58:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 15:58:26.459337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1174/2341] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:58:26 2022] POST /identity/v3/projects => generated 383 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:26.460500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65d360-db81-47df-a902-c9989d3fdee3 service swift] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:26.476051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65d360-db81-47df-a902-c9989d3fdee3 service swift] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:26.517972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:26.569924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266e408-42fd-4291-ae31-8c493f4a538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:26.570817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266e408-42fd-4291-ae31-8c493f4a538b admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:26.571176 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266e408-42fd-4291-ae31-8c493f4a538b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:26.571526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266e408-42fd-4291-ae31-8c493f4a538b admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:26.572307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266e408-42fd-4291-ae31-8c493f4a538b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:26.575063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266e408-42fd-4291-ae31-8c493f4a538b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:26.674850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1168/2342] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:58:26 2022] POST /identity/v3/users => generated 298 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:26.678658 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0f19-cd62-4f4e-bd07-976829d5c345 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:26.689749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0f19-cd62-4f4e-bd07-976829d5c345 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:26.695735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00f7c4-1218-43ee-acb3-ad75628a8ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:26.698864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00f7c4-1218-43ee-acb3-ad75628a8ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:26.698864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00f7c4-1218-43ee-acb3-ad75628a8ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:26.698864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00f7c4-1218-43ee-acb3-ad75628a8ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:26.698864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00f7c4-1218-43ee-acb3-ad75628a8ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:26.700383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00f7c4-1218-43ee-acb3-ad75628a8ab4 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:26.713999 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266e408-42fd-4291-ae31-8c493f4a538b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:26.716353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c266e408-42fd-4291-ae31-8c493f4a538b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:26.717954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1175/2343] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:26.729397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00f7c4-1218-43ee-acb3-ad75628a8ab4 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:26.745430 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00f7c4-1218-43ee-acb3-ad75628a8ab4 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:26.746249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266e408-42fd-4291-ae31-8c493f4a538b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:26.752307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace001c3-be2c-4d02-bfbf-99e52ec43c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:26.754268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace001c3-be2c-4d02-bfbf-99e52ec43c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:26.756441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace001c3-be2c-4d02-bfbf-99e52ec43c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:26.756441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace001c3-be2c-4d02-bfbf-99e52ec43c00 admin admin] PATH_INFO: `/v3/projects/974c0f39a8394fc595ec631524887425/users/81e50606e0d841b99a2be4d1f81a96d2/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:26.757467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:26.758383 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:26.758677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:26.758962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] PATH_INFO: `/v3/users/13035fc79c124492930fca0a48db6551` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:26.764032 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13035fc79c124492930fca0a48db6551)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:26.766017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:26.774266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace001c3-be2c-4d02-bfbf-99e52ec43c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974c0f39a8394fc595ec631524887425, user_id=81e50606e0d841b99a2be4d1f81a96d2, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:26.777717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace001c3-be2c-4d02-bfbf-99e52ec43c00 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:26.870414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1176/2344] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:26 2022] PUT /identity/v3/projects/974c0f39a8394fc595ec631524887425/users/81e50606e0d841b99a2be4d1f81a96d2/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:26.882469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace001c3-be2c-4d02-bfbf-99e52ec43c00 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:26.959984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace001c3-be2c-4d02-bfbf-99e52ec43c00 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:26.964870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6587719-70fc-450b-9454-c2dba3ed7331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:26.965590 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6587719-70fc-450b-9454-c2dba3ed7331 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:26.965880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6587719-70fc-450b-9454-c2dba3ed7331 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:26.966160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6587719-70fc-450b-9454-c2dba3ed7331 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:26.966899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6587719-70fc-450b-9454-c2dba3ed7331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:26.968568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6587719-70fc-450b-9454-c2dba3ed7331 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:26.986981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1177/2345] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:58:26.991543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6587719-70fc-450b-9454-c2dba3ed7331 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:27.009633 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6587719-70fc-450b-9454-c2dba3ed7331 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:27.015843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9cbf3-9aff-4c1d-8a33-f9e0e642bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:27.017192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9cbf3-9aff-4c1d-8a33-f9e0e642bafd admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.017483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9cbf3-9aff-4c1d-8a33-f9e0e642bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.018440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9cbf3-9aff-4c1d-8a33-f9e0e642bafd admin admin] PATH_INFO: `/v3/projects/974c0f39a8394fc595ec631524887425/users/81e50606e0d841b99a2be4d1f81a96d2/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.020199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9cbf3-9aff-4c1d-8a33-f9e0e642bafd admin admin] RBAC: Authorizing `identity:create_grant(project_id=974c0f39a8394fc595ec631524887425, user_id=81e50606e0d841b99a2be4d1f81a96d2, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:27.025783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9cbf3-9aff-4c1d-8a33-f9e0e642bafd admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:27.032941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13035fc79c124492930fca0a48db6551'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:27.033660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13035fc79c124492930fca0a48db6551'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:27.046310 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1178/2346] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:26 2022] PUT /identity/v3/projects/974c0f39a8394fc595ec631524887425/users/81e50606e0d841b99a2be4d1f81a96d2/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:27.054869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9cbf3-9aff-4c1d-8a33-f9e0e642bafd admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:27.057930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-73a9cbf3-9aff-4c1d-8a33-f9e0e642bafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.061405 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13035fc79c124492930fca0a48db6551'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:27.062324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13035fc79c124492930fca0a48db6551'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:27.070787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c24b5825-7a67-453a-9670-68714f72cf58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13035fc79c124492930fca0a48db6551'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:27.074573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:27.129689 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9cbf3-9aff-4c1d-8a33-f9e0e642bafd admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:27.134435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:27.135328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.135589 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.136016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] PATH_INFO: `/v3/users/f1f4967e10be436ba976b60a50185b20` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.138189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f4967e10be436ba976b60a50185b20)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:27.140284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:27.172375 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1169/2347] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:26 2022] DELETE /identity/v3/users/13035fc79c124492930fca0a48db6551 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:58:27.173981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70287480-12f5-43df-b37a-3ccf77a2313f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.174322 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70287480-12f5-43df-b37a-3ccf77a2313f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.175191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70287480-12f5-43df-b37a-3ccf77a2313f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.209793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-70287480-12f5-43df-b37a-3ccf77a2313f None None] MFA Rules not processed for user `81e50606e0d841b99a2be4d1f81a96d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:27.211093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-70287480-12f5-43df-b37a-3ccf77a2313f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.300124 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1170/2348] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:27.301522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa23f-e66c-41eb-a4dd-5db27d12e535 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.301809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa23f-e66c-41eb-a4dd-5db27d12e535 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.302088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa23f-e66c-41eb-a4dd-5db27d12e535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.322362 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-80dfa23f-e66c-41eb-a4dd-5db27d12e535 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:27.323675 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-80dfa23f-e66c-41eb-a4dd-5db27d12e535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.343364 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:27.379005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f4967e10be436ba976b60a50185b20'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:27.380400 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f4967e10be436ba976b60a50185b20'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:27.381271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f4967e10be436ba976b60a50185b20'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:27.393319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f4967e10be436ba976b60a50185b20'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:27.402763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1171/2349] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:27.403341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfa23f-e66c-41eb-a4dd-5db27d12e535 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:27.404703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-80dfa23f-e66c-41eb-a4dd-5db27d12e535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.407926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f4967e10be436ba976b60a50185b20'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:27.467542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfa23f-e66c-41eb-a4dd-5db27d12e535 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:27.471259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f741a-5111-4d2f-9d58-516e5035b48a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:27.471919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f741a-5111-4d2f-9d58-516e5035b48a service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.472211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f741a-5111-4d2f-9d58-516e5035b48a service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.472626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f741a-5111-4d2f-9d58-516e5035b48a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.474590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b68f741a-5111-4d2f-9d58-516e5035b48a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.490192 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1179/2350] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:27 2022] DELETE /identity/v3/users/f1f4967e10be436ba976b60a50185b20 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:27.491752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:27.494298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.497469 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f741a-5111-4d2f-9d58-516e5035b48a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:27.498915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f741a-5111-4d2f-9d58-516e5035b48a service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:27.511988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1172/2351] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:27.515544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77c273-0cfd-4902-9d03-e5bda7df9e64 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.519650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77c273-0cfd-4902-9d03-e5bda7df9e64 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.519650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77c273-0cfd-4902-9d03-e5bda7df9e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.538050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6a77c273-0cfd-4902-9d03-e5bda7df9e64 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:27.539284 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6a77c273-0cfd-4902-9d03-e5bda7df9e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.593598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee7817-16c0-41f4-a915-5320da7af843 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:27.597683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c2243-9037-4254-9e1c-d6dc5a1493fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:27.600647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c2243-9037-4254-9e1c-d6dc5a1493fc service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.600926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c2243-9037-4254-9e1c-d6dc5a1493fc service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.605609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c2243-9037-4254-9e1c-d6dc5a1493fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.645246 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1173/2352] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:27.647194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae96d7-340c-4c26-8089-f3954cc04e94 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.649143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae96d7-340c-4c26-8089-f3954cc04e94 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.649143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae96d7-340c-4c26-8089-f3954cc04e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.675325 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a1ae96d7-340c-4c26-8089-f3954cc04e94 None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:27.676720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a1ae96d7-340c-4c26-8089-f3954cc04e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.682473 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c2243-9037-4254-9e1c-d6dc5a1493fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:27.684244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c2243-9037-4254-9e1c-d6dc5a1493fc service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:27.701680 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1180/2353] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:27.703044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c2243-9037-4254-9e1c-d6dc5a1493fc service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:27.757978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1174/2354] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 23 15:58:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:27.758837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae96d7-340c-4c26-8089-f3954cc04e94 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:27.766267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a1ae96d7-340c-4c26-8089-f3954cc04e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.779334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c2243-9037-4254-9e1c-d6dc5a1493fc service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:27.779520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae96d7-340c-4c26-8089-f3954cc04e94 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:27.783473 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00631597-d0bf-4c55-93f9-9cf0d4143501 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:27.783853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed6a1f-2da9-47cc-bc01-c60ba850a583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:27.784135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00631597-d0bf-4c55-93f9-9cf0d4143501 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.784453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00631597-d0bf-4c55-93f9-9cf0d4143501 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.784735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00631597-d0bf-4c55-93f9-9cf0d4143501 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.784851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed6a1f-2da9-47cc-bc01-c60ba850a583 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.785164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed6a1f-2da9-47cc-bc01-c60ba850a583 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.785456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed6a1f-2da9-47cc-bc01-c60ba850a583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.786742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-00631597-d0bf-4c55-93f9-9cf0d4143501 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.796224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed6a1f-2da9-47cc-bc01-c60ba850a583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:27.797888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed6a1f-2da9-47cc-bc01-c60ba850a583 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:27.809357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1175/2355] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:27.810102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed6a1f-2da9-47cc-bc01-c60ba850a583 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:27.811678 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-21ed6a1f-2da9-47cc-bc01-c60ba850a583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:27.844295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:27.878447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00631597-d0bf-4c55-93f9-9cf0d4143501 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:27.880497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00631597-d0bf-4c55-93f9-9cf0d4143501 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:27.921211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1181/2356] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:27.938912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed6a1f-2da9-47cc-bc01-c60ba850a583 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:27.956758 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd306ab-3529-456b-b3dc-dfc343d4a381 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:27.957692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd306ab-3529-456b-b3dc-dfc343d4a381 service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:27.958199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd306ab-3529-456b-b3dc-dfc343d4a381 service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:27.958730 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd306ab-3529-456b-b3dc-dfc343d4a381 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:27.985727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd306ab-3529-456b-b3dc-dfc343d4a381 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:27.987651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd306ab-3529-456b-b3dc-dfc343d4a381 service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:28.021482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1176/2357] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 15:58:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:58:28.469092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00631597-d0bf-4c55-93f9-9cf0d4143501 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:28.471285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-00631597-d0bf-4c55-93f9-9cf0d4143501 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:28.485186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd306ab-3529-456b-b3dc-dfc343d4a381 service swift] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:28.486688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6bd306ab-3529-456b-b3dc-dfc343d4a381 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:28.508937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd306ab-3529-456b-b3dc-dfc343d4a381 service swift] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:28.514397 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:28.515277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:28.515594 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:28.515844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] PATH_INFO: `/v3/projects/bd07b008154d412a856b5a381680b5a9` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:28.517779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd07b008154d412a856b5a381680b5a9)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:28.519882 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:28.522776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00631597-d0bf-4c55-93f9-9cf0d4143501 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:28.540169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:28.545366 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:28.547094 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:28.548607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] PATH_INFO: `/v3/projects/85aedee3f4624ebca7a05c03afc26b90` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:28.551698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85aedee3f4624ebca7a05c03afc26b90)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:28.555901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:28.690521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85aedee3f4624ebca7a05c03afc26b90', 'request_id': 'req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:28.696820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd07b008154d412a856b5a381680b5a9', 'request_id': 'req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:28.696989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85aedee3f4624ebca7a05c03afc26b90', 'request_id': 'req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:28.700117 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:28.702807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85aedee3f4624ebca7a05c03afc26b90', 'request_id': 'req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:28.706161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd07b008154d412a856b5a381680b5a9', 'request_id': 'req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:28.711654 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd07b008154d412a856b5a381680b5a9', 'request_id': 'req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:28.789782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] The token cache is being invalidate because project 85aedee3f4624ebca7a05c03afc26b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:28.790075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:28.792435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ca41f6bc-98c8-43fd-a991-5e9ea24d22d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:28.793924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1182/2358] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:28 2022] DELETE /identity/v3/projects/85aedee3f4624ebca7a05c03afc26b90 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:28.809854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532909-0c4f-4784-ade8-bec3013fabe7 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:28.810095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532909-0c4f-4784-ade8-bec3013fabe7 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:28.810382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532909-0c4f-4784-ade8-bec3013fabe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:28.861072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-87532909-0c4f-4784-ade8-bec3013fabe7 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:28.862363 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-87532909-0c4f-4784-ade8-bec3013fabe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:28.925242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] The token cache is being invalidate because project bd07b008154d412a856b5a381680b5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:28.928341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:28.930499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-673d48e7-71c5-4ba9-9a93-60aec0c3ef5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:28.937095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1177/2359] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:28 2022] DELETE /identity/v3/projects/bd07b008154d412a856b5a381680b5a9 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:58:28.940672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e110-8e5b-49ae-a68e-6f83453641ef None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:28.941150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e110-8e5b-49ae-a68e-6f83453641ef None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:28.942091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e110-8e5b-49ae-a68e-6f83453641ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:28.952893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1183/2360] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:28.967662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9c66f-d634-4a62-813e-50e9a33908ab None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:28.968527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9c66f-d634-4a62-813e-50e9a33908ab None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:28.968527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9c66f-d634-4a62-813e-50e9a33908ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:28.984221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6c70e110-8e5b-49ae-a68e-6f83453641ef None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:28.985614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6c70e110-8e5b-49ae-a68e-6f83453641ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:28.996861 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1178/2361] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:28.999927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3923fb-e0f8-4c75-bc1b-4ff9a1dd0a71 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.000560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3923fb-e0f8-4c75-bc1b-4ff9a1dd0a71 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.000986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3923fb-e0f8-4c75-bc1b-4ff9a1dd0a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.006035 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-38e9c66f-d634-4a62-813e-50e9a33908ab None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:29.007332 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-38e9c66f-d634-4a62-813e-50e9a33908ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:29.025242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1d3923fb-e0f8-4c75-bc1b-4ff9a1dd0a71 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:29.026506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1d3923fb-e0f8-4c75-bc1b-4ff9a1dd0a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:29.035889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1179/2362] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:29.037077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe40e76-2a37-4f58-bd4a-af2f5a18efef None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.037368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe40e76-2a37-4f58-bd4a-af2f5a18efef None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.037645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe40e76-2a37-4f58-bd4a-af2f5a18efef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.057916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ffe40e76-2a37-4f58-bd4a-af2f5a18efef None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:29.059128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ffe40e76-2a37-4f58-bd4a-af2f5a18efef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:29.067550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1180/2363] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:29.068549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe40e76-2a37-4f58-bd4a-af2f5a18efef None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.078955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe40e76-2a37-4f58-bd4a-af2f5a18efef None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.085000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1184/2364] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 15:58:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:29.088149 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9c66f-d634-4a62-813e-50e9a33908ab None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.094732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17ee14-a1a6-4ff8-855d-20604a8daf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.095536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17ee14-a1a6-4ff8-855d-20604a8daf06 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.097807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17ee14-a1a6-4ff8-855d-20604a8daf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.098141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17ee14-a1a6-4ff8-855d-20604a8daf06 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.098899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17ee14-a1a6-4ff8-855d-20604a8daf06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:58:29.099277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17ee14-a1a6-4ff8-855d-20604a8daf06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.100939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17ee14-a1a6-4ff8-855d-20604a8daf06 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.121939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1181/2365] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:58:29.121939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17ee14-a1a6-4ff8-855d-20604a8daf06 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.122188 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9c66f-d634-4a62-813e-50e9a33908ab None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.126628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0939373-0f20-4a44-a12b-a184f1b0e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.132692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dd17ee14-a1a6-4ff8-855d-20604a8daf06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:29.132835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0939373-0f20-4a44-a12b-a184f1b0e95e admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.132835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0939373-0f20-4a44-a12b-a184f1b0e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.132835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0939373-0f20-4a44-a12b-a184f1b0e95e admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.132835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0939373-0f20-4a44-a12b-a184f1b0e95e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:58:29.132835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0939373-0f20-4a44-a12b-a184f1b0e95e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.132835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0939373-0f20-4a44-a12b-a184f1b0e95e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.148799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1185/2366] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:58:29.149722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0939373-0f20-4a44-a12b-a184f1b0e95e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.162943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0939373-0f20-4a44-a12b-a184f1b0e95e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.168406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c828a-5000-498e-a779-40a9dd7ba008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.169160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c828a-5000-498e-a779-40a9dd7ba008 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.170047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c828a-5000-498e-a779-40a9dd7ba008 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.170341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c828a-5000-498e-a779-40a9dd7ba008 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.172723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c828a-5000-498e-a779-40a9dd7ba008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.173180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c828a-5000-498e-a779-40a9dd7ba008 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.218260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17ee14-a1a6-4ff8-855d-20604a8daf06 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.223354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17cbaa-d963-439c-aa72-77e2d8eb3a67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.224005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17cbaa-d963-439c-aa72-77e2d8eb3a67 service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.224318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17cbaa-d963-439c-aa72-77e2d8eb3a67 service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.224683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17cbaa-d963-439c-aa72-77e2d8eb3a67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.234366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17cbaa-d963-439c-aa72-77e2d8eb3a67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.236019 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17cbaa-d963-439c-aa72-77e2d8eb3a67 service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.246328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1182/2367] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 15:58:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:58:29.247263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17cbaa-d963-439c-aa72-77e2d8eb3a67 service swift] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.262966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17cbaa-d963-439c-aa72-77e2d8eb3a67 service swift] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.267751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53ba2b-810d-461c-8a25-32b24605ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.270327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53ba2b-810d-461c-8a25-32b24605ff89 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.270327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53ba2b-810d-461c-8a25-32b24605ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.270327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53ba2b-810d-461c-8a25-32b24605ff89 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.270327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53ba2b-810d-461c-8a25-32b24605ff89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.272237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53ba2b-810d-461c-8a25-32b24605ff89 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.272789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1186/2368] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:58:29 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:29.276330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c828a-5000-498e-a779-40a9dd7ba008 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.342694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c828a-5000-498e-a779-40a9dd7ba008 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.346897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:29.350591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd74fc-fb2c-4d29-a55b-732f68831b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.350591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd74fc-fb2c-4d29-a55b-732f68831b8c admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.350591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd74fc-fb2c-4d29-a55b-732f68831b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.350591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd74fc-fb2c-4d29-a55b-732f68831b8c admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.351047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd74fc-fb2c-4d29-a55b-732f68831b8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.351735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd74fc-fb2c-4d29-a55b-732f68831b8c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.406271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1183/2369] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:58:29 2022] POST /identity/v3/projects => generated 389 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:29.410466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53ba2b-810d-461c-8a25-32b24605ff89 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.417847 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:29.452473 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1187/2370] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:58:29 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:58:29.455455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd74fc-fb2c-4d29-a55b-732f68831b8c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.471391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd74fc-fb2c-4d29-a55b-732f68831b8c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.487391 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53ba2b-810d-461c-8a25-32b24605ff89 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.493788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f2f0-696f-4ccb-aab8-58616ad969d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.494486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f2f0-696f-4ccb-aab8-58616ad969d3 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.494775 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f2f0-696f-4ccb-aab8-58616ad969d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.496161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f2f0-696f-4ccb-aab8-58616ad969d3 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.497153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0f2f0-696f-4ccb-aab8-58616ad969d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.500555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0f2f0-696f-4ccb-aab8-58616ad969d3 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.538475 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f4846-580d-4d43-a239-3877655273ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.539157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f4846-580d-4d43-a239-3877655273ff admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.539472 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f4846-580d-4d43-a239-3877655273ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.539725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f4846-580d-4d43-a239-3877655273ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.540511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f4846-580d-4d43-a239-3877655273ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.542640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f4846-580d-4d43-a239-3877655273ff admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.554901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1188/2371] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:29.563854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f4846-580d-4d43-a239-3877655273ff admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.574148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f4846-580d-4d43-a239-3877655273ff admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.578797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde8f7e-892b-4326-9679-d3c1e2265ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.579508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde8f7e-892b-4326-9679-d3c1e2265ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.579797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde8f7e-892b-4326-9679-d3c1e2265ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.580072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde8f7e-892b-4326-9679-d3c1e2265ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.580836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde8f7e-892b-4326-9679-d3c1e2265ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.582423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde8f7e-892b-4326-9679-d3c1e2265ee5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.592458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1189/2372] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:29.598939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde8f7e-892b-4326-9679-d3c1e2265ee5 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.609443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde8f7e-892b-4326-9679-d3c1e2265ee5 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.614179 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001e259-ed5b-4f9b-9f3e-0691a26452de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.615083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001e259-ed5b-4f9b-9f3e-0691a26452de admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.615348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001e259-ed5b-4f9b-9f3e-0691a26452de admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.615626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001e259-ed5b-4f9b-9f3e-0691a26452de admin admin] PATH_INFO: `/v3/projects/bfd605bd3d014c72ad5f0751015ad51d/users/29ed5e8b4af6436196259912983b4939/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.636263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001e259-ed5b-4f9b-9f3e-0691a26452de admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd605bd3d014c72ad5f0751015ad51d, user_id=29ed5e8b4af6436196259912983b4939, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.639211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001e259-ed5b-4f9b-9f3e-0691a26452de admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.666577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1184/2373] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:58:29 2022] POST /identity/v3/users => generated 295 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:29.669864 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e44c92-fd61-4e2c-80eb-9b45dfdaee17 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.670391 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e44c92-fd61-4e2c-80eb-9b45dfdaee17 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.670719 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e44c92-fd61-4e2c-80eb-9b45dfdaee17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.691101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c1e44c92-fd61-4e2c-80eb-9b45dfdaee17 None None] MFA Rules not processed for user `5b52924339de4cc08fe35dbf2a5fdcd0`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:29.692757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c1e44c92-fd61-4e2c-80eb-9b45dfdaee17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:29.716334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1190/2374] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:29 2022] PUT /identity/v3/projects/bfd605bd3d014c72ad5f0751015ad51d/users/29ed5e8b4af6436196259912983b4939/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:58:29.716334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001e259-ed5b-4f9b-9f3e-0691a26452de admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.760324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1185/2375] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:29.760324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3739b73-3eb3-493f-a1d0-e4037279699a None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.760324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3739b73-3eb3-493f-a1d0-e4037279699a None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.760324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3739b73-3eb3-493f-a1d0-e4037279699a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.817256 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-d3739b73-3eb3-493f-a1d0-e4037279699a None None] MFA Rules not processed for user `29ed5e8b4af6436196259912983b4939`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:29.818632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d3739b73-3eb3-493f-a1d0-e4037279699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:29.851291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001e259-ed5b-4f9b-9f3e-0691a26452de admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.861592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed7d6a-af54-426f-8865-b303ab6b309f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.862775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed7d6a-af54-426f-8865-b303ab6b309f admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.863702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed7d6a-af54-426f-8865-b303ab6b309f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.865189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed7d6a-af54-426f-8865-b303ab6b309f admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.867757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed7d6a-af54-426f-8865-b303ab6b309f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.871467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed7d6a-af54-426f-8865-b303ab6b309f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.888411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1191/2376] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:29.894079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed7d6a-af54-426f-8865-b303ab6b309f admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.911673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed7d6a-af54-426f-8865-b303ab6b309f admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:29.922408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a667a-4e5e-4c76-8602-93996f130081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:29.923215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a667a-4e5e-4c76-8602-93996f130081 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:29.923549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a667a-4e5e-4c76-8602-93996f130081 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:29.924229 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a667a-4e5e-4c76-8602-93996f130081 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:29.925106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a667a-4e5e-4c76-8602-93996f130081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:29.929121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1186/2377] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:29.940892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a667a-4e5e-4c76-8602-93996f130081 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:29.960604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3739b73-3eb3-493f-a1d0-e4037279699a None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:29.970687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d3739b73-3eb3-493f-a1d0-e4037279699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:29.978962 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1192/2378] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:29.980099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a667a-4e5e-4c76-8602-93996f130081 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:30.003338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a667a-4e5e-4c76-8602-93996f130081 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:30.018769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6e66d-3273-45d9-b6fb-78b320df0a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:30.019811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6e66d-3273-45d9-b6fb-78b320df0a97 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:30.020202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6e66d-3273-45d9-b6fb-78b320df0a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:30.020927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6e66d-3273-45d9-b6fb-78b320df0a97 admin admin] PATH_INFO: `/v3/projects/b1c8725c83f0497597120658a06a56bf/users/27850a07aaac4a6ca2c9520d79b3f06e/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:30.043939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6e66d-3273-45d9-b6fb-78b320df0a97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c8725c83f0497597120658a06a56bf, user_id=27850a07aaac4a6ca2c9520d79b3f06e, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:30.046146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6e66d-3273-45d9-b6fb-78b320df0a97 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:30.059497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3739b73-3eb3-493f-a1d0-e4037279699a None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:30.064897 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c74cd-d1d3-4740-87d4-419fc6e6f100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:30.065710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c74cd-d1d3-4740-87d4-419fc6e6f100 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:30.066914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c74cd-d1d3-4740-87d4-419fc6e6f100 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:30.066914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c74cd-d1d3-4740-87d4-419fc6e6f100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:30.090992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c74cd-d1d3-4740-87d4-419fc6e6f100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:30.090992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c74cd-d1d3-4740-87d4-419fc6e6f100 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:30.101564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1187/2379] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:58:30.103238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017b1f9-1b76-49ae-aee1-41c1bafa9b61 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:30.103766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017b1f9-1b76-49ae-aee1-41c1bafa9b61 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:30.104240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017b1f9-1b76-49ae-aee1-41c1bafa9b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:30.130984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e017b1f9-1b76-49ae-aee1-41c1bafa9b61 None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:30.132555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e017b1f9-1b76-49ae-aee1-41c1bafa9b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:30.196882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1193/2380] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:29 2022] PUT /identity/v3/projects/b1c8725c83f0497597120658a06a56bf/users/27850a07aaac4a6ca2c9520d79b3f06e/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:30.210065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62459f4-626c-436d-b232-b34109d297fa None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:30.210065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62459f4-626c-436d-b232-b34109d297fa None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:30.210257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62459f4-626c-436d-b232-b34109d297fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:30.226261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1188/2381] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 23 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:30.243019 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017b1f9-1b76-49ae-aee1-41c1bafa9b61 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:30.251079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-c62459f4-626c-436d-b232-b34109d297fa None None] MFA Rules not processed for user `27850a07aaac4a6ca2c9520d79b3f06e`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:30.253603 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c62459f4-626c-436d-b232-b34109d297fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:30.366424 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017b1f9-1b76-49ae-aee1-41c1bafa9b61 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:30.414013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1194/2382] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:30.426105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62459f4-626c-436d-b232-b34109d297fa None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:30.427877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c62459f4-626c-436d-b232-b34109d297fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:30.463396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a35a52-7669-4fe3-9230-324a98567471 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:30.464148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a35a52-7669-4fe3-9230-324a98567471 service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:30.464462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a35a52-7669-4fe3-9230-324a98567471 service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:30.464748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a35a52-7669-4fe3-9230-324a98567471 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:30.508703 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62459f4-626c-436d-b232-b34109d297fa None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:30.514822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92196da-9509-4b84-a5ac-d095e69e6ee3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:30.515798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92196da-9509-4b84-a5ac-d095e69e6ee3 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:30.516238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92196da-9509-4b84-a5ac-d095e69e6ee3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:30.516707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92196da-9509-4b84-a5ac-d095e69e6ee3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:30.530070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92196da-9509-4b84-a5ac-d095e69e6ee3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:30.532120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a35a52-7669-4fe3-9230-324a98567471 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:30.532708 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92196da-9509-4b84-a5ac-d095e69e6ee3 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:30.534130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a35a52-7669-4fe3-9230-324a98567471 service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:30.544569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1195/2383] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:58:30.545805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1189/2384] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 15:58:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 15:58:31.103313 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:31.209600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:31.535874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a35a52-7669-4fe3-9230-324a98567471 service swift] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:31.537452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d1a35a52-7669-4fe3-9230-324a98567471 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:31.643464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a35a52-7669-4fe3-9230-324a98567471 service swift] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:31.677247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:31.678486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:31.678965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:31.679528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] PATH_INFO: `/v3/users/3aa2ed15b110472c84989304e86947af` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:31.681207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa2ed15b110472c84989304e86947af)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:31.685067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:31.930633 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa2ed15b110472c84989304e86947af'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:31.933606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa2ed15b110472c84989304e86947af'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:31.959834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa2ed15b110472c84989304e86947af'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:31.961280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa2ed15b110472c84989304e86947af'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:31.969065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa2ed15b110472c84989304e86947af'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:32.078060 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1190/2385] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:31 2022] DELETE /identity/v3/users/3aa2ed15b110472c84989304e86947af => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:32.082612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab786b2-53ed-4b27-816b-6aaf325a1b67 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:32.083121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab786b2-53ed-4b27-816b-6aaf325a1b67 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:32.083244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab786b2-53ed-4b27-816b-6aaf325a1b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:32.102087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-1ab786b2-53ed-4b27-816b-6aaf325a1b67 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:32.103299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1ab786b2-53ed-4b27-816b-6aaf325a1b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:32.182924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1196/2386] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:32.193700 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:32.195224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:32.329330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f88c69-b85a-4c3f-998a-7e60c86ebfe5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:32.332544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f73104-0615-4563-9cf6-2ff979d5b1d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:32.333232 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f73104-0615-4563-9cf6-2ff979d5b1d5 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:32.333990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f73104-0615-4563-9cf6-2ff979d5b1d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:32.334363 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f73104-0615-4563-9cf6-2ff979d5b1d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:32.345414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f73104-0615-4563-9cf6-2ff979d5b1d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:32.346942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f73104-0615-4563-9cf6-2ff979d5b1d5 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:32.358291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1191/2387] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:32.434434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:32.563458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:32.826217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab786b2-53ed-4b27-816b-6aaf325a1b67 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:32.839024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1ab786b2-53ed-4b27-816b-6aaf325a1b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:32.885972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:32.890214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab786b2-53ed-4b27-816b-6aaf325a1b67 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:32.915325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:32.919303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:32.919606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:32.919890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] PATH_INFO: `/v3/projects/a807c9a5c0f540509dd8ed3e2ac1506d` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:32.921989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=a807c9a5c0f540509dd8ed3e2ac1506d)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:32.924397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:33.066224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a807c9a5c0f540509dd8ed3e2ac1506d', 'request_id': 'req-ad7b6e21-76fd-4861-82fb-418c18044eca'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:33.073865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a807c9a5c0f540509dd8ed3e2ac1506d', 'request_id': 'req-ad7b6e21-76fd-4861-82fb-418c18044eca'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:33.080720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a807c9a5c0f540509dd8ed3e2ac1506d', 'request_id': 'req-ad7b6e21-76fd-4861-82fb-418c18044eca'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:33.164254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] The token cache is being invalidate because project a807c9a5c0f540509dd8ed3e2ac1506d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:33.164923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:33.165173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:33.166765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1197/2388] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:32 2022] DELETE /identity/v3/projects/a807c9a5c0f540509dd8ed3e2ac1506d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:58:33.170733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f73104-0615-4563-9cf6-2ff979d5b1d5 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:33.180231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f73104-0615-4563-9cf6-2ff979d5b1d5 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:33.247715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:33.248626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:33.248911 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:33.249188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] PATH_INFO: `/v3/users/81e50606e0d841b99a2be4d1f81a96d2` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:33.250384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81e50606e0d841b99a2be4d1f81a96d2)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:33.252312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:33.493839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81e50606e0d841b99a2be4d1f81a96d2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:33.495139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81e50606e0d841b99a2be4d1f81a96d2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:33.518254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:33.518402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81e50606e0d841b99a2be4d1f81a96d2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:33.518402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81e50606e0d841b99a2be4d1f81a96d2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:33.526167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-b86d4ce6-59af-4bbe-b499-ab1505143989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81e50606e0d841b99a2be4d1f81a96d2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:33.594130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1192/2389] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:33 2022] DELETE /identity/v3/users/81e50606e0d841b99a2be4d1f81a96d2 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:34.052010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:34.053980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:34.087511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b6e21-76fd-4861-82fb-418c18044eca admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:34.149082 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:34.149996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:34.150350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:34.151253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] PATH_INFO: `/v3/projects/974c0f39a8394fc595ec631524887425` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:34.152568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=974c0f39a8394fc595ec631524887425)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:34.154615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:34.302307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '974c0f39a8394fc595ec631524887425', 'request_id': 'req-0070d07e-df55-4574-ba14-7efb8b1c11c4'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:34.308720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '974c0f39a8394fc595ec631524887425', 'request_id': 'req-0070d07e-df55-4574-ba14-7efb8b1c11c4'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:34.314602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '974c0f39a8394fc595ec631524887425', 'request_id': 'req-0070d07e-df55-4574-ba14-7efb8b1c11c4'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:34.425332 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] The token cache is being invalidate because project 974c0f39a8394fc595ec631524887425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:34.425620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:34.426342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-0070d07e-df55-4574-ba14-7efb8b1c11c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:34.428476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1198/2390] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:34 2022] DELETE /identity/v3/projects/974c0f39a8394fc595ec631524887425 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:34.433989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d5cf-c53a-444a-8b07-80e50ab968e5 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:34.434487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d5cf-c53a-444a-8b07-80e50ab968e5 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:34.434583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d5cf-c53a-444a-8b07-80e50ab968e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:34.440840 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:34.454227 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-0929d5cf-c53a-444a-8b07-80e50ab968e5 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:34.455423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0929d5cf-c53a-444a-8b07-80e50ab968e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:34.522735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1193/2391] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:34.530483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8002655-8868-4d48-aeb8-8905a966eb64 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:34.530888 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8002655-8868-4d48-aeb8-8905a966eb64 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:34.531055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8002655-8868-4d48-aeb8-8905a966eb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:34.573718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a8002655-8868-4d48-aeb8-8905a966eb64 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:34.573718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a8002655-8868-4d48-aeb8-8905a966eb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:34.585014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1199/2392] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:34.586270 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d5cf-c53a-444a-8b07-80e50ab968e5 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:34.596663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d5cf-c53a-444a-8b07-80e50ab968e5 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:34.601609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a230c-327f-4e66-a693-c153360e9bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:34.602276 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a230c-327f-4e66-a693-c153360e9bed admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:34.602562 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a230c-327f-4e66-a693-c153360e9bed admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:34.602849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a230c-327f-4e66-a693-c153360e9bed admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:34.603632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a230c-327f-4e66-a693-c153360e9bed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:58:34.603957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a230c-327f-4e66-a693-c153360e9bed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:34.605591 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a230c-327f-4e66-a693-c153360e9bed admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:34.620247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1194/2393] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:58:34.624508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8002655-8868-4d48-aeb8-8905a966eb64 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:34.634736 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8002655-8868-4d48-aeb8-8905a966eb64 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:34.640219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4101c1f-4a9e-4b53-ae87-806f169f8a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:34.640921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4101c1f-4a9e-4b53-ae87-806f169f8a2a admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:34.643773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4101c1f-4a9e-4b53-ae87-806f169f8a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:34.643773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4101c1f-4a9e-4b53-ae87-806f169f8a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:34.643773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4101c1f-4a9e-4b53-ae87-806f169f8a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:34.644252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4101c1f-4a9e-4b53-ae87-806f169f8a2a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:34.737497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1200/2394] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:58:34 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:34.741658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a230c-327f-4e66-a693-c153360e9bed admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:34.752576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a230c-327f-4e66-a693-c153360e9bed admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:34.817381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404f828-f3db-460d-84b8-5a235f30b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:34.818317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404f828-f3db-460d-84b8-5a235f30b3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:34.818583 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404f828-f3db-460d-84b8-5a235f30b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:34.818937 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404f828-f3db-460d-84b8-5a235f30b3b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:34.819837 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404f828-f3db-460d-84b8-5a235f30b3b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:34.822243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404f828-f3db-460d-84b8-5a235f30b3b7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:34.918118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1195/2395] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:58:34 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:34.919411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4101c1f-4a9e-4b53-ae87-806f169f8a2a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:34.929805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4101c1f-4a9e-4b53-ae87-806f169f8a2a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:34.935131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ac976-237b-4c2b-928a-cec600fb0e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:34.935814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ac976-237b-4c2b-928a-cec600fb0e9c admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:34.936171 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ac976-237b-4c2b-928a-cec600fb0e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:34.936398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ac976-237b-4c2b-928a-cec600fb0e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:34.937631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ac976-237b-4c2b-928a-cec600fb0e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:34.939253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ac976-237b-4c2b-928a-cec600fb0e9c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:34.949926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1201/2396] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:34.954115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404f828-f3db-460d-84b8-5a235f30b3b7 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:34.964223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404f828-f3db-460d-84b8-5a235f30b3b7 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:34.968902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcfe1b-d08a-4a40-b526-1532e9fa267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:34.969571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcfe1b-d08a-4a40-b526-1532e9fa267b admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:34.969853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcfe1b-d08a-4a40-b526-1532e9fa267b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:34.970662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcfe1b-d08a-4a40-b526-1532e9fa267b admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:34.970902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcfe1b-d08a-4a40-b526-1532e9fa267b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:34.972446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcfe1b-d08a-4a40-b526-1532e9fa267b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:34.986275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1196/2397] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:34.987408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ac976-237b-4c2b-928a-cec600fb0e9c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:34.997965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ac976-237b-4c2b-928a-cec600fb0e9c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:35.003996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd37c54-ba24-4475-8b2c-01500e0b5e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:35.005563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd37c54-ba24-4475-8b2c-01500e0b5e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:35.006006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd37c54-ba24-4475-8b2c-01500e0b5e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:35.006378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd37c54-ba24-4475-8b2c-01500e0b5e52 admin admin] PATH_INFO: `/v3/projects/d8081f532ab24033a2ae8b0fbefe4d98/users/115cbccd3b014209bfc12047e0406ae3/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:35.026061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd37c54-ba24-4475-8b2c-01500e0b5e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8081f532ab24033a2ae8b0fbefe4d98, user_id=115cbccd3b014209bfc12047e0406ae3, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:35.029122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd37c54-ba24-4475-8b2c-01500e0b5e52 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:35.114284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1202/2398] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:34 2022] PUT /identity/v3/projects/d8081f532ab24033a2ae8b0fbefe4d98/users/115cbccd3b014209bfc12047e0406ae3/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:35.126251 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3969700-ea28-4ef7-8b66-f121521e625d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:35.126565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3969700-ea28-4ef7-8b66-f121521e625d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:35.126878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3969700-ea28-4ef7-8b66-f121521e625d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:35.155683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c3969700-ea28-4ef7-8b66-f121521e625d None None] MFA Rules not processed for user `115cbccd3b014209bfc12047e0406ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:35.156952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c3969700-ea28-4ef7-8b66-f121521e625d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:35.228784 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1197/2399] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:35.240636 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd37c54-ba24-4475-8b2c-01500e0b5e52 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:35.242390 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6bd37c54-ba24-4475-8b2c-01500e0b5e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:35.272458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:35.306182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd37c54-ba24-4475-8b2c-01500e0b5e52 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:35.310169 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce289b8-d3e3-42e5-9eef-0b346c8ab81a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:35.310912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce289b8-d3e3-42e5-9eef-0b346c8ab81a service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:35.311194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce289b8-d3e3-42e5-9eef-0b346c8ab81a service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:35.311625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce289b8-d3e3-42e5-9eef-0b346c8ab81a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:35.320325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce289b8-d3e3-42e5-9eef-0b346c8ab81a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:35.322021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce289b8-d3e3-42e5-9eef-0b346c8ab81a service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:35.330827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1203/2400] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:35.356175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3969700-ea28-4ef7-8b66-f121521e625d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:35.420578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3969700-ea28-4ef7-8b66-f121521e625d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:35.425450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:35.426352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:35.426638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:35.426917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] PATH_INFO: `/v3/users/115cbccd3b014209bfc12047e0406ae3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:35.428188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=115cbccd3b014209bfc12047e0406ae3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:35.430184 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:35.437020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:35.626373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '115cbccd3b014209bfc12047e0406ae3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:35.627130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '115cbccd3b014209bfc12047e0406ae3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:35.649237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '115cbccd3b014209bfc12047e0406ae3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:35.649237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '115cbccd3b014209bfc12047e0406ae3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:35.649495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '115cbccd3b014209bfc12047e0406ae3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:35.764424 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1198/2401] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:35 2022] DELETE /identity/v3/users/115cbccd3b014209bfc12047e0406ae3 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:58:35.768920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b65c-41c4-464d-992a-dd752e8730c6 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:35.769456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b65c-41c4-464d-992a-dd752e8730c6 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:35.769744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b65c-41c4-464d-992a-dd752e8730c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:35.787699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-f392b65c-41c4-464d-992a-dd752e8730c6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:35.788916 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f392b65c-41c4-464d-992a-dd752e8730c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:35.849514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1204/2402] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:35.859135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:35.860930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:35.920111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bdd52-24b9-4c54-804b-f639dace9a58 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:35.923872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99c14b-c3e3-4ce6-be89-3e5e191659ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:35.924552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99c14b-c3e3-4ce6-be89-3e5e191659ad service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:35.924839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99c14b-c3e3-4ce6-be89-3e5e191659ad service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:35.925115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99c14b-c3e3-4ce6-be89-3e5e191659ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:35.934901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99c14b-c3e3-4ce6-be89-3e5e191659ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:35.936323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99c14b-c3e3-4ce6-be89-3e5e191659ad service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:35.946128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1199/2403] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:36.320041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392b65c-41c4-464d-992a-dd752e8730c6 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:36.321520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f392b65c-41c4-464d-992a-dd752e8730c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:36.334659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392b65c-41c4-464d-992a-dd752e8730c6 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:36.339604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:36.340752 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:36.341234 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:36.341695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] PATH_INFO: `/v3/projects/d8081f532ab24033a2ae8b0fbefe4d98` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:36.343028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8081f532ab24033a2ae8b0fbefe4d98)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:36.345270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:36.462007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8081f532ab24033a2ae8b0fbefe4d98', 'request_id': 'req-b0f7cb99-52a4-4984-8d8c-7505878df287'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:36.468603 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8081f532ab24033a2ae8b0fbefe4d98', 'request_id': 'req-b0f7cb99-52a4-4984-8d8c-7505878df287'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:36.475949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8081f532ab24033a2ae8b0fbefe4d98', 'request_id': 'req-b0f7cb99-52a4-4984-8d8c-7505878df287'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:36.547050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] The token cache is being invalidate because project d8081f532ab24033a2ae8b0fbefe4d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:36.547341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:36.547987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b0f7cb99-52a4-4984-8d8c-7505878df287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:36.549529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1205/2404] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:36 2022] DELETE /identity/v3/projects/d8081f532ab24033a2ae8b0fbefe4d98 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:36.558453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f4fb3-0c3c-492b-b23f-9fd15e03d700 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:36.558982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f4fb3-0c3c-492b-b23f-9fd15e03d700 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:36.558982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f4fb3-0c3c-492b-b23f-9fd15e03d700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:36.576640 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-f64f4fb3-0c3c-492b-b23f-9fd15e03d700 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:36.577839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f64f4fb3-0c3c-492b-b23f-9fd15e03d700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:36.590705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:36.637880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1200/2405] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:36.646299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a38952-56ea-480b-8abe-488be8ad3bb9 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:36.646918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a38952-56ea-480b-8abe-488be8ad3bb9 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:36.647288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a38952-56ea-480b-8abe-488be8ad3bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:36.669382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-91a38952-56ea-480b-8abe-488be8ad3bb9 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:36.672563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-91a38952-56ea-480b-8abe-488be8ad3bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:36.679174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1206/2406] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:36.691749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f4fb3-0c3c-492b-b23f-9fd15e03d700 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:36.701991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f4fb3-0c3c-492b-b23f-9fd15e03d700 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:36.706892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309581a-e4c8-4261-86c9-0dcb23461e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:36.707556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309581a-e4c8-4261-86c9-0dcb23461e0c admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:36.711487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309581a-e4c8-4261-86c9-0dcb23461e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:36.711487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309581a-e4c8-4261-86c9-0dcb23461e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:36.711487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309581a-e4c8-4261-86c9-0dcb23461e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:58:36.711487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309581a-e4c8-4261-86c9-0dcb23461e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:36.711487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309581a-e4c8-4261-86c9-0dcb23461e0c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:36.724571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1201/2407] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:58:36.729726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a38952-56ea-480b-8abe-488be8ad3bb9 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:36.740350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a38952-56ea-480b-8abe-488be8ad3bb9 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:36.746064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84350a-69c6-4274-90ad-492f275731ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:36.746799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84350a-69c6-4274-90ad-492f275731ec admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:36.747087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84350a-69c6-4274-90ad-492f275731ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:36.747365 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84350a-69c6-4274-90ad-492f275731ec admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:36.748062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84350a-69c6-4274-90ad-492f275731ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:36.750105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84350a-69c6-4274-90ad-492f275731ec admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:36.826523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1207/2408] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:58:36 2022] POST /identity/v3/projects => generated 399 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:58:36.830880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309581a-e4c8-4261-86c9-0dcb23461e0c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:36.841373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309581a-e4c8-4261-86c9-0dcb23461e0c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:36.904935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b4644-8421-4867-9208-c7182deab1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:36.905681 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b4644-8421-4867-9208-c7182deab1ba admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:36.905972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b4644-8421-4867-9208-c7182deab1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:36.906251 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b4644-8421-4867-9208-c7182deab1ba admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:36.906962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b4644-8421-4867-9208-c7182deab1ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:36.908930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b4644-8421-4867-9208-c7182deab1ba admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:37.014235 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1202/2409] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:58:36 2022] POST /identity/v3/users => generated 300 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:37.027092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84350a-69c6-4274-90ad-492f275731ec admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:37.040196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84350a-69c6-4274-90ad-492f275731ec admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:37.044055 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00837025-1753-4779-96d0-fd156fe4add3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:37.044801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00837025-1753-4779-96d0-fd156fe4add3 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:37.045060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00837025-1753-4779-96d0-fd156fe4add3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:37.045337 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00837025-1753-4779-96d0-fd156fe4add3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:37.046102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00837025-1753-4779-96d0-fd156fe4add3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:37.047842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00837025-1753-4779-96d0-fd156fe4add3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:37.059207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1208/2410] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:58:37.063983 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b4644-8421-4867-9208-c7182deab1ba admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:37.076728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b4644-8421-4867-9208-c7182deab1ba admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:37.082069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210b355-7a08-4823-9956-c45fb8c8dd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:37.083024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210b355-7a08-4823-9956-c45fb8c8dd8b admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:37.083331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210b355-7a08-4823-9956-c45fb8c8dd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:37.083657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210b355-7a08-4823-9956-c45fb8c8dd8b admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:37.084320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210b355-7a08-4823-9956-c45fb8c8dd8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:37.086034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210b355-7a08-4823-9956-c45fb8c8dd8b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:37.100212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1203/2411] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:58:37.101430 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00837025-1753-4779-96d0-fd156fe4add3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:37.111793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00837025-1753-4779-96d0-fd156fe4add3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:37.116464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c729d0d-d7de-4803-bd33-ed057506a970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:37.117342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c729d0d-d7de-4803-bd33-ed057506a970 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:37.117627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c729d0d-d7de-4803-bd33-ed057506a970 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:37.117902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c729d0d-d7de-4803-bd33-ed057506a970 admin admin] PATH_INFO: `/v3/projects/d847dc8c437047f58f8b60f04b868787/users/088fad4a1f6a46c884fd2b90382adb99/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:37.121321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:37.132887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c729d0d-d7de-4803-bd33-ed057506a970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d847dc8c437047f58f8b60f04b868787, user_id=088fad4a1f6a46c884fd2b90382adb99, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:37.136087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c729d0d-d7de-4803-bd33-ed057506a970 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:37.646394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1209/2412] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:37 2022] PUT /identity/v3/projects/d847dc8c437047f58f8b60f04b868787/users/088fad4a1f6a46c884fd2b90382adb99/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:37.656631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714509f-8d58-46fa-ad1b-6911b1b54fdd None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:37.657164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714509f-8d58-46fa-ad1b-6911b1b54fdd None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:37.657522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714509f-8d58-46fa-ad1b-6911b1b54fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:37.683640 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2714509f-8d58-46fa-ad1b-6911b1b54fdd None None] MFA Rules not processed for user `088fad4a1f6a46c884fd2b90382adb99`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:37.684922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2714509f-8d58-46fa-ad1b-6911b1b54fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:37.752125 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1204/2413] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:37.796680 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c729d0d-d7de-4803-bd33-ed057506a970 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:37.798485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5c729d0d-d7de-4803-bd33-ed057506a970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:37.860822 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c729d0d-d7de-4803-bd33-ed057506a970 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:37.864977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76865e-0919-4f7e-8b2e-18001ca4bcb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:37.865701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76865e-0919-4f7e-8b2e-18001ca4bcb1 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:37.866062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76865e-0919-4f7e-8b2e-18001ca4bcb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:37.866412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76865e-0919-4f7e-8b2e-18001ca4bcb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:37.875660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76865e-0919-4f7e-8b2e-18001ca4bcb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:37.877583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76865e-0919-4f7e-8b2e-18001ca4bcb1 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:37.886890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1210/2414] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:58:41.131396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714509f-8d58-46fa-ad1b-6911b1b54fdd None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:41.133096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2714509f-8d58-46fa-ad1b-6911b1b54fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:41.154027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:41.220040 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714509f-8d58-46fa-ad1b-6911b1b54fdd None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:41.225012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:41.226051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:41.226406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:41.226753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] PATH_INFO: `/v3/users/8f8bc5cf16844078b77c6072205d380f` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:41.228073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8bc5cf16844078b77c6072205d380f)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:41.230709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:41.450354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8bc5cf16844078b77c6072205d380f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:41.451062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8bc5cf16844078b77c6072205d380f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:41.465783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8bc5cf16844078b77c6072205d380f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:41.466497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8bc5cf16844078b77c6072205d380f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:41.473706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8bc5cf16844078b77c6072205d380f'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:41.575622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1205/2415] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:41 2022] DELETE /identity/v3/users/8f8bc5cf16844078b77c6072205d380f => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:41.581793 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1432e-67fc-4a37-a393-7822b82bfbcf None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:41.582229 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1432e-67fc-4a37-a393-7822b82bfbcf None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:41.582340 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1432e-67fc-4a37-a393-7822b82bfbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:41.602415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-56a1432e-67fc-4a37-a393-7822b82bfbcf None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:41.604229 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-56a1432e-67fc-4a37-a393-7822b82bfbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:41.675734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1211/2416] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:41.685260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:41.686889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:41.778300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbac399-e45a-4ecc-9055-30ed4bae3da1 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:41.783411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c17f8-da32-4bf6-80c8-33350e07a81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:41.784303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c17f8-da32-4bf6-80c8-33350e07a81f service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:41.784779 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c17f8-da32-4bf6-80c8-33350e07a81f service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:41.785398 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c17f8-da32-4bf6-80c8-33350e07a81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:41.811187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c17f8-da32-4bf6-80c8-33350e07a81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:41.812750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c17f8-da32-4bf6-80c8-33350e07a81f service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:41.835670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1206/2417] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:42.323755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1432e-67fc-4a37-a393-7822b82bfbcf None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:42.328870 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-56a1432e-67fc-4a37-a393-7822b82bfbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:42.349572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1432e-67fc-4a37-a393-7822b82bfbcf None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:42.359959 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:42.361014 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:42.361309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:42.361592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] PATH_INFO: `/v3/projects/2a73a241a6ec473b8b16890fe997ba4e` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:42.362860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a73a241a6ec473b8b16890fe997ba4e)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:42.364882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:42.485264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a73a241a6ec473b8b16890fe997ba4e', 'request_id': 'req-64dfd026-04ee-447f-8652-9bccae509308'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:42.507799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a73a241a6ec473b8b16890fe997ba4e', 'request_id': 'req-64dfd026-04ee-447f-8652-9bccae509308'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:42.510104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a73a241a6ec473b8b16890fe997ba4e', 'request_id': 'req-64dfd026-04ee-447f-8652-9bccae509308'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:42.612251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] The token cache is being invalidate because project 2a73a241a6ec473b8b16890fe997ba4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:42.612812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:42.613381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-64dfd026-04ee-447f-8652-9bccae509308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:42.615398 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1212/2418] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:42 2022] DELETE /identity/v3/projects/2a73a241a6ec473b8b16890fe997ba4e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:51.820493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=72,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:52.608662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:53.698012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c17f8-da32-4bf6-80c8-33350e07a81f service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:53.699458 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d04c17f8-da32-4bf6-80c8-33350e07a81f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:53.712765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c17f8-da32-4bf6-80c8-33350e07a81f service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:53.779213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:53.783497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:53.784783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:53.785063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] PATH_INFO: `/v3/users/29ed5e8b4af6436196259912983b4939` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:53.787820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ed5e8b4af6436196259912983b4939)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:53.789809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:54.069183 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ed5e8b4af6436196259912983b4939'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:54.070435 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ed5e8b4af6436196259912983b4939'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:54.082694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ed5e8b4af6436196259912983b4939'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:54.083365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ed5e8b4af6436196259912983b4939'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:54.090476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ed5e8b4af6436196259912983b4939'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:54.185410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1207/2419] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:53 2022] DELETE /identity/v3/users/29ed5e8b4af6436196259912983b4939 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:54.196950 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7637b4-bd93-46f4-8d6a-1a6e651f0848 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:54.197464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7637b4-bd93-46f4-8d6a-1a6e651f0848 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:54.197722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7637b4-bd93-46f4-8d6a-1a6e651f0848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:54.222458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ae7637b4-bd93-46f4-8d6a-1a6e651f0848 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:54.225593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ae7637b4-bd93-46f4-8d6a-1a6e651f0848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:54.301382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1213/2420] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 15:58:54.311742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:54.313956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:54.372455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033a32e-bd4f-4a87-933e-d49019528a7d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:54.376117 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffd90d-d2e1-4249-9f4e-354a86166463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:54.376935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd90d-d2e1-4249-9f4e-354a86166463 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:54.377226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd90d-d2e1-4249-9f4e-354a86166463 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:54.377509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd90d-d2e1-4249-9f4e-354a86166463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:54.386672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffd90d-d2e1-4249-9f4e-354a86166463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:54.388053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffd90d-d2e1-4249-9f4e-354a86166463 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:54.398424 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1208/2421] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:54.427522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7637b4-bd93-46f4-8d6a-1a6e651f0848 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:54.428748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ae7637b4-bd93-46f4-8d6a-1a6e651f0848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:54.441399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7637b4-bd93-46f4-8d6a-1a6e651f0848 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:54.446551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:54.447425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:54.447704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:54.447984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] PATH_INFO: `/v3/users/088fad4a1f6a46c884fd2b90382adb99` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:54.449402 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=088fad4a1f6a46c884fd2b90382adb99)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:54.451550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:54.642904 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088fad4a1f6a46c884fd2b90382adb99'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:54.643374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088fad4a1f6a46c884fd2b90382adb99'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:54.644073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088fad4a1f6a46c884fd2b90382adb99'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:54.652313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088fad4a1f6a46c884fd2b90382adb99'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:54.665778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088fad4a1f6a46c884fd2b90382adb99'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:54.799188 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1214/2422] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:54 2022] DELETE /identity/v3/users/088fad4a1f6a46c884fd2b90382adb99 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:54.803611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff0e11-ae35-474c-86ab-4b96dfdae667 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:54.804214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff0e11-ae35-474c-86ab-4b96dfdae667 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:54.804375 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff0e11-ae35-474c-86ab-4b96dfdae667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:54.822851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-0dff0e11-ae35-474c-86ab-4b96dfdae667 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:54.824210 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0dff0e11-ae35-474c-86ab-4b96dfdae667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:54.844927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:54.876415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:54.878808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:54.902040 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1209/2423] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:54.910599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff0e11-ae35-474c-86ab-4b96dfdae667 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:54.912166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0dff0e11-ae35-474c-86ab-4b96dfdae667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:54.960689 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37461a0-7789-4bf5-a814-c224091cf746 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:54.965369 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:54.966226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:54.966507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:54.966784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] PATH_INFO: `/v3/projects/bfd605bd3d014c72ad5f0751015ad51d` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:54.967956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd605bd3d014c72ad5f0751015ad51d)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:54.969897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:54.973003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff0e11-ae35-474c-86ab-4b96dfdae667 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:54.977316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c503e23-c068-4415-9edc-7fecc70b5e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:54.982331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c503e23-c068-4415-9edc-7fecc70b5e8d service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:54.982331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c503e23-c068-4415-9edc-7fecc70b5e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:54.982331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c503e23-c068-4415-9edc-7fecc70b5e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:54.988215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c503e23-c068-4415-9edc-7fecc70b5e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:54.988567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:54.990075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c503e23-c068-4415-9edc-7fecc70b5e8d service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:55.000126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1210/2424] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:58:55.113381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd605bd3d014c72ad5f0751015ad51d', 'request_id': 'req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:55.119525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd605bd3d014c72ad5f0751015ad51d', 'request_id': 'req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:55.124920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd605bd3d014c72ad5f0751015ad51d', 'request_id': 'req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:55.231986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] The token cache is being invalidate because project bfd605bd3d014c72ad5f0751015ad51d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:55.232634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:55.233506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3a9910fb-c7a2-465b-b1da-0e0cacde8b10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:55.243967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1215/2425] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:54 2022] DELETE /identity/v3/projects/bfd605bd3d014c72ad5f0751015ad51d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:58:55.249160 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05c69-e737-489a-9787-1c8a20e3f253 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:55.249453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05c69-e737-489a-9787-1c8a20e3f253 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:55.249796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05c69-e737-489a-9787-1c8a20e3f253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:55.293513 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-0fb05c69-e737-489a-9787-1c8a20e3f253 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:55.294767 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0fb05c69-e737-489a-9787-1c8a20e3f253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:55.401144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1211/2426] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:55.406415 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c12c77-d873-4e2b-8e0a-f5e4e07ec8e4 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:55.407559 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c12c77-d873-4e2b-8e0a-f5e4e07ec8e4 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:55.408439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c12c77-d873-4e2b-8e0a-f5e4e07ec8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:55.430030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d8c12c77-d873-4e2b-8e0a-f5e4e07ec8e4 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:55.431254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d8c12c77-d873-4e2b-8e0a-f5e4e07ec8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:55.440393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1216/2427] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:55.444254 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb05c69-e737-489a-9787-1c8a20e3f253 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:55.460843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb05c69-e737-489a-9787-1c8a20e3f253 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:55.463523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130da0b3-548c-446f-9ac7-73b4275f1df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:55.464239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130da0b3-548c-446f-9ac7-73b4275f1df0 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:55.464582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130da0b3-548c-446f-9ac7-73b4275f1df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:55.464892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130da0b3-548c-446f-9ac7-73b4275f1df0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:55.465657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130da0b3-548c-446f-9ac7-73b4275f1df0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:58:55.465968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130da0b3-548c-446f-9ac7-73b4275f1df0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:55.468648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130da0b3-548c-446f-9ac7-73b4275f1df0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:55.487695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1212/2428] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:58:55.488975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c12c77-d873-4e2b-8e0a-f5e4e07ec8e4 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:55.500927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c12c77-d873-4e2b-8e0a-f5e4e07ec8e4 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:55.508268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0c84-5646-424c-b56b-ac753e556bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:55.509531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0c84-5646-424c-b56b-ac753e556bca admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:55.510282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0c84-5646-424c-b56b-ac753e556bca admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:55.511228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0c84-5646-424c-b56b-ac753e556bca admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:55.512640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff0c84-5646-424c-b56b-ac753e556bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:55.516320 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff0c84-5646-424c-b56b-ac753e556bca admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:55.580308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130da0b3-548c-446f-9ac7-73b4275f1df0 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:55.582172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-130da0b3-548c-446f-9ac7-73b4275f1df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:55.609764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1217/2429] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:58:55 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:58:55.614004 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0c84-5646-424c-b56b-ac753e556bca admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:55.614123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130da0b3-548c-446f-9ac7-73b4275f1df0 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:55.676133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:55.689272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:55.689502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0c84-5646-424c-b56b-ac753e556bca admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:55.691263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:55.691263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:55.691650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] PATH_INFO: `/v3/projects/d847dc8c437047f58f8b60f04b868787` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:55.693470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d847dc8c437047f58f8b60f04b868787)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:55.699507 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:55.699631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e39bd-37bf-465e-a6c0-85f628cadcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:55.699631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e39bd-37bf-465e-a6c0-85f628cadcf2 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:55.699631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e39bd-37bf-465e-a6c0-85f628cadcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:55.699631 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e39bd-37bf-465e-a6c0-85f628cadcf2 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:55.699991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e39bd-37bf-465e-a6c0-85f628cadcf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:55.700846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e39bd-37bf-465e-a6c0-85f628cadcf2 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:55.791455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1218/2430] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:58:55 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:58:55.794434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e39bd-37bf-465e-a6c0-85f628cadcf2 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:55.805252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e39bd-37bf-465e-a6c0-85f628cadcf2 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:55.810207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1f109-22f7-46dd-9577-9b5fa3dc5e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:55.810928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1f109-22f7-46dd-9577-9b5fa3dc5e5a admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:55.811214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1f109-22f7-46dd-9577-9b5fa3dc5e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:55.811502 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1f109-22f7-46dd-9577-9b5fa3dc5e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:55.812230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1f109-22f7-46dd-9577-9b5fa3dc5e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:55.813872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1f109-22f7-46dd-9577-9b5fa3dc5e5a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:55.822479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd847dc8c437047f58f8b60f04b868787', 'request_id': 'req-5b854ec1-fe84-4e19-b538-8fb11d6da611'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:55.823977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1219/2431] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:58:55.828669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1f109-22f7-46dd-9577-9b5fa3dc5e5a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:55.832312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd847dc8c437047f58f8b60f04b868787', 'request_id': 'req-5b854ec1-fe84-4e19-b538-8fb11d6da611'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:55.838738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd847dc8c437047f58f8b60f04b868787', 'request_id': 'req-5b854ec1-fe84-4e19-b538-8fb11d6da611'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:55.847990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1f109-22f7-46dd-9577-9b5fa3dc5e5a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:55.888798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:55.897955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a6586-c0a6-455e-8ae1-d4ca8260b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:55.898769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a6586-c0a6-455e-8ae1-d4ca8260b7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:55.899053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a6586-c0a6-455e-8ae1-d4ca8260b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:55.899329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a6586-c0a6-455e-8ae1-d4ca8260b7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:55.900049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a6586-c0a6-455e-8ae1-d4ca8260b7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:55.901683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a6586-c0a6-455e-8ae1-d4ca8260b7d9 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:55.914170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1220/2432] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:58:55.916682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a6586-c0a6-455e-8ae1-d4ca8260b7d9 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:55.926315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a6586-c0a6-455e-8ae1-d4ca8260b7d9 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:55.931567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89de68-ca78-4f8a-af59-d00766537f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:55.932474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89de68-ca78-4f8a-af59-d00766537f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:55.932779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89de68-ca78-4f8a-af59-d00766537f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:55.933060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89de68-ca78-4f8a-af59-d00766537f49 admin admin] PATH_INFO: `/v3/projects/53d22d201cab4638bf0ae9392fd79233/users/36e2bae836f5409f822af1f2919e4b80/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:55.954041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89de68-ca78-4f8a-af59-d00766537f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d22d201cab4638bf0ae9392fd79233, user_id=36e2bae836f5409f822af1f2919e4b80, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:55.954041 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89de68-ca78-4f8a-af59-d00766537f49 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:55.971930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] The token cache is being invalidate because project d847dc8c437047f58f8b60f04b868787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:55.972273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:55.972938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-5b854ec1-fe84-4e19-b538-8fb11d6da611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:55.974501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1213/2433] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:55 2022] DELETE /identity/v3/projects/d847dc8c437047f58f8b60f04b868787 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:58:56.059865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1221/2434] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:58:55 2022] PUT /identity/v3/projects/53d22d201cab4638bf0ae9392fd79233/users/36e2bae836f5409f822af1f2919e4b80/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:58:56.065171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567695e-1d15-47d3-a2a8-f8b5d07541ec None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:56.065547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567695e-1d15-47d3-a2a8-f8b5d07541ec None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:56.065895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567695e-1d15-47d3-a2a8-f8b5d07541ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:56.102357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-1567695e-1d15-47d3-a2a8-f8b5d07541ec None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:56.103575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1567695e-1d15-47d3-a2a8-f8b5d07541ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:56.175684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1214/2435] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:56.185264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89de68-ca78-4f8a-af59-d00766537f49 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:56.186993 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1d89de68-ca78-4f8a-af59-d00766537f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:56.246536 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89de68-ca78-4f8a-af59-d00766537f49 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:56.250252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf513781-426c-4f39-b22b-73c0ad959a73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:56.250899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf513781-426c-4f39-b22b-73c0ad959a73 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:56.251384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf513781-426c-4f39-b22b-73c0ad959a73 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:56.251660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf513781-426c-4f39-b22b-73c0ad959a73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:56.262392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf513781-426c-4f39-b22b-73c0ad959a73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:56.264803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf513781-426c-4f39-b22b-73c0ad959a73 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:56.276054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1222/2436] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:56.447822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567695e-1d15-47d3-a2a8-f8b5d07541ec None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:56.449350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1567695e-1d15-47d3-a2a8-f8b5d07541ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:56.462994 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567695e-1d15-47d3-a2a8-f8b5d07541ec None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:56.468248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:56.469287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:56.469569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:56.469842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] PATH_INFO: `/v3/users/27850a07aaac4a6ca2c9520d79b3f06e` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:56.471133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] RBAC: Authorizing `identity:delete_user(user_id=27850a07aaac4a6ca2c9520d79b3f06e)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:56.473113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:56.683839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27850a07aaac4a6ca2c9520d79b3f06e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:56.684658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27850a07aaac4a6ca2c9520d79b3f06e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:56.698945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27850a07aaac4a6ca2c9520d79b3f06e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:56.699686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27850a07aaac4a6ca2c9520d79b3f06e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:56.707461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27850a07aaac4a6ca2c9520d79b3f06e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:56.712044 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:56.825060 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1215/2437] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:58:56 2022] DELETE /identity/v3/users/27850a07aaac4a6ca2c9520d79b3f06e => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:56.830693 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee789e-136c-4cc0-b0d4-3dc8b2d12851 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:56.831229 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee789e-136c-4cc0-b0d4-3dc8b2d12851 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:56.831918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee789e-136c-4cc0-b0d4-3dc8b2d12851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:56.851468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-fcee789e-136c-4cc0-b0d4-3dc8b2d12851 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:56.852830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-fcee789e-136c-4cc0-b0d4-3dc8b2d12851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:56.929347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1223/2438] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:56.938856 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:56.942486 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:57.013257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4bff1-dab1-4942-8f78-9fb7b671c21b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:57.017156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048ce84-5768-495f-b1ee-432a1c4b6657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:57.017847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048ce84-5768-495f-b1ee-432a1c4b6657 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:57.018130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048ce84-5768-495f-b1ee-432a1c4b6657 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:57.018408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048ce84-5768-495f-b1ee-432a1c4b6657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:57.029671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048ce84-5768-495f-b1ee-432a1c4b6657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:57.031487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048ce84-5768-495f-b1ee-432a1c4b6657 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:57.044616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1216/2439] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:58:57.573673 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee789e-136c-4cc0-b0d4-3dc8b2d12851 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:57.575295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-fcee789e-136c-4cc0-b0d4-3dc8b2d12851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:57.598922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee789e-136c-4cc0-b0d4-3dc8b2d12851 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:57.606256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:57.607171 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:57.607470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:57.607727 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] PATH_INFO: `/v3/projects/b1c8725c83f0497597120658a06a56bf` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:57.609300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c8725c83f0497597120658a06a56bf)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:57.611985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:57.682904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fb2e4-2415-4b9f-969a-07a8588e4f7d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:57.683192 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fb2e4-2415-4b9f-969a-07a8588e4f7d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:57.683471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fb2e4-2415-4b9f-969a-07a8588e4f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:57.715509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e46fb2e4-2415-4b9f-969a-07a8588e4f7d None None] MFA Rules not processed for user `36e2bae836f5409f822af1f2919e4b80`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:57.717336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e46fb2e4-2415-4b9f-969a-07a8588e4f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:57.761638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c8725c83f0497597120658a06a56bf', 'request_id': 'req-55c6c498-47e1-49ab-90fb-654072ab9a88'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:57.773980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c8725c83f0497597120658a06a56bf', 'request_id': 'req-55c6c498-47e1-49ab-90fb-654072ab9a88'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:57.783923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c8725c83f0497597120658a06a56bf', 'request_id': 'req-55c6c498-47e1-49ab-90fb-654072ab9a88'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:57.787279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:57.832989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1217/2440] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:58:57.841306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fb2e4-2415-4b9f-969a-07a8588e4f7d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:58:57.842715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e46fb2e4-2415-4b9f-969a-07a8588e4f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:57.908091 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fb2e4-2415-4b9f-969a-07a8588e4f7d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:58:57.920167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] The token cache is being invalidate because project b1c8725c83f0497597120658a06a56bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:58:57.920481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:57.921239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-55c6c498-47e1-49ab-90fb-654072ab9a88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:58:57.940855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1224/2441] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:58:57 2022] DELETE /identity/v3/projects/b1c8725c83f0497597120658a06a56bf => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:58:57.968187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:58:57.978670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3135b55-0f2c-46cf-be90-60f271b2896b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:58:57.980453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3135b55-0f2c-46cf-be90-60f271b2896b service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:57.980453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3135b55-0f2c-46cf-be90-60f271b2896b service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:57.980453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3135b55-0f2c-46cf-be90-60f271b2896b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:57.982846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f3135b55-0f2c-46cf-be90-60f271b2896b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:58.081881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3135b55-0f2c-46cf-be90-60f271b2896b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:58:58.083481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3135b55-0f2c-46cf-be90-60f271b2896b service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:58:58.101156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1218/2442] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:58:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:58:59.859777 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7f0f0-902e-4e73-8a3e-5eeb45832d70 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:58:59.862790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7f0f0-902e-4e73-8a3e-5eeb45832d70 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:58:59.862790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7f0f0-902e-4e73-8a3e-5eeb45832d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:58:59.880510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-39c7f0f0-902e-4e73-8a3e-5eeb45832d70 None None] MFA Rules not processed for user `36e2bae836f5409f822af1f2919e4b80`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:58:59.881779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-39c7f0f0-902e-4e73-8a3e-5eeb45832d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:58:59.888866 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1225/2443] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:06.399527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da8306-1728-41c2-85ba-234b507ed328 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:06.399527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da8306-1728-41c2-85ba-234b507ed328 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:06.399527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da8306-1728-41c2-85ba-234b507ed328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:06.458264 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e4da8306-1728-41c2-85ba-234b507ed328 None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:06.459607 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e4da8306-1728-41c2-85ba-234b507ed328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:06.534898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1219/2444] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 15:59:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:06.548330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7f0f0-902e-4e73-8a3e-5eeb45832d70 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:06.549935 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-39c7f0f0-902e-4e73-8a3e-5eeb45832d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:06.658827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7f0f0-902e-4e73-8a3e-5eeb45832d70 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:06.664122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91465acf-f7ec-400d-a044-31037e5c5798 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:06.664804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91465acf-f7ec-400d-a044-31037e5c5798 service swift] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:06.665085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91465acf-f7ec-400d-a044-31037e5c5798 service swift] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:06.665356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91465acf-f7ec-400d-a044-31037e5c5798 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:06.673874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91465acf-f7ec-400d-a044-31037e5c5798 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:06.675454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91465acf-f7ec-400d-a044-31037e5c5798 service swift] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:06.685302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1226/2445] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 15:59:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:59:10.517865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da8306-1728-41c2-85ba-234b507ed328 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:10.519655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e4da8306-1728-41c2-85ba-234b507ed328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:10.614056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:10.633306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da8306-1728-41c2-85ba-234b507ed328 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:10.641894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b761345-f5cb-4a5e-b3b3-31765413e605 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:10.642853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b761345-f5cb-4a5e-b3b3-31765413e605 service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:10.643324 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b761345-f5cb-4a5e-b3b3-31765413e605 service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:10.643703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b761345-f5cb-4a5e-b3b3-31765413e605 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:10.645813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0b761345-f5cb-4a5e-b3b3-31765413e605 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:10.661749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b761345-f5cb-4a5e-b3b3-31765413e605 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:10.663948 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b761345-f5cb-4a5e-b3b3-31765413e605 service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:10.683259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1220/2446] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:59:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:59:16.682252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=46 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:20.402094 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91465acf-f7ec-400d-a044-31037e5c5798 service swift] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:20.404757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-91465acf-f7ec-400d-a044-31037e5c5798 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:20.484991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91465acf-f7ec-400d-a044-31037e5c5798 service swift] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:20.489840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:20.490758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:20.491046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:20.491327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] PATH_INFO: `/v3/users/14411a9a02c5492b8878dedd1e3e775d` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:20.492675 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=14411a9a02c5492b8878dedd1e3e775d)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:20.494722 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:20.675392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=9 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:20.759137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14411a9a02c5492b8878dedd1e3e775d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:20.759778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14411a9a02c5492b8878dedd1e3e775d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:20.761124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14411a9a02c5492b8878dedd1e3e775d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:20.768349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14411a9a02c5492b8878dedd1e3e775d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:20.780185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14411a9a02c5492b8878dedd1e3e775d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:20.856896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b167b-c25f-4646-8793-9d2e15d7bffa None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:20.857249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b167b-c25f-4646-8793-9d2e15d7bffa None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:20.857543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b167b-c25f-4646-8793-9d2e15d7bffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:20.861656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1227/2447] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:59:20 2022] DELETE /identity/v3/users/14411a9a02c5492b8878dedd1e3e775d => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:59:20.879707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-619b167b-c25f-4646-8793-9d2e15d7bffa None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:20.883237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-619b167b-c25f-4646-8793-9d2e15d7bffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:20.962482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1221/2448] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:20.973998 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:20.975642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:21.052644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dddda-b41d-4188-ab28-189aedbe83ba admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:21.056856 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1eb885-4350-4ce6-aee6-31e241412eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:21.057844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eb885-4350-4ce6-aee6-31e241412eea service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:21.058266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eb885-4350-4ce6-aee6-31e241412eea service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:21.058756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1eb885-4350-4ce6-aee6-31e241412eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:21.070954 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1eb885-4350-4ce6-aee6-31e241412eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:21.073528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1eb885-4350-4ce6-aee6-31e241412eea service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:21.092387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1228/2449] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 15:59:21.559686 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b167b-c25f-4646-8793-9d2e15d7bffa None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:21.561424 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-619b167b-c25f-4646-8793-9d2e15d7bffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:21.579292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b167b-c25f-4646-8793-9d2e15d7bffa None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:21.584547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:21.585559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:21.585925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:21.586393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] PATH_INFO: `/v3/projects/5df2f448827740aa84665e079f469100` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:21.589347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df2f448827740aa84665e079f469100)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:21.593239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:21.725120 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df2f448827740aa84665e079f469100', 'request_id': 'req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:21.732252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df2f448827740aa84665e079f469100', 'request_id': 'req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:21.738354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df2f448827740aa84665e079f469100', 'request_id': 'req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:21.819796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] The token cache is being invalidate because project 5df2f448827740aa84665e079f469100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:59:21.820173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:21.820949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d48c1f2-7eda-42dd-9f78-ec191e3adf80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:21.822846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1222/2450] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:59:21 2022] DELETE /identity/v3/projects/5df2f448827740aa84665e079f469100 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:59:21.828316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e0fb5-d69d-4af3-8f2d-1560489ebadb None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:21.833800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e0fb5-d69d-4af3-8f2d-1560489ebadb None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:21.833800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e0fb5-d69d-4af3-8f2d-1560489ebadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:21.858639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-971e0fb5-d69d-4af3-8f2d-1560489ebadb None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:21.860083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-971e0fb5-d69d-4af3-8f2d-1560489ebadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:21.882756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:21.938885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1229/2451] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:21.940777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e71cb-d0c7-4ac1-9684-f3605d0e4f90 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:21.941094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e71cb-d0c7-4ac1-9684-f3605d0e4f90 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:21.941455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e71cb-d0c7-4ac1-9684-f3605d0e4f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:21.962261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-786e71cb-d0c7-4ac1-9684-f3605d0e4f90 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:21.965637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-786e71cb-d0c7-4ac1-9684-f3605d0e4f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:21.975070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1223/2452] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:21.979775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e0fb5-d69d-4af3-8f2d-1560489ebadb None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:21.990177 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e0fb5-d69d-4af3-8f2d-1560489ebadb None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:21.996135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aea7e-1000-45f6-91ad-fc73a905d48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:21.997189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aea7e-1000-45f6-91ad-fc73a905d48a admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:21.997807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aea7e-1000-45f6-91ad-fc73a905d48a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:21.997995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aea7e-1000-45f6-91ad-fc73a905d48a admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:21.999043 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aea7e-1000-45f6-91ad-fc73a905d48a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:59:21.999206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aea7e-1000-45f6-91ad-fc73a905d48a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:22.001251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aea7e-1000-45f6-91ad-fc73a905d48a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:22.017285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1230/2453] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 15:59:22.022035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e71cb-d0c7-4ac1-9684-f3605d0e4f90 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:22.033498 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e71cb-d0c7-4ac1-9684-f3605d0e4f90 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:22.039371 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea9f61-ed76-45c4-a00a-ad49eaca8867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:22.040353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea9f61-ed76-45c4-a00a-ad49eaca8867 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:22.040665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea9f61-ed76-45c4-a00a-ad49eaca8867 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:22.040945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea9f61-ed76-45c4-a00a-ad49eaca8867 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:22.042435 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea9f61-ed76-45c4-a00a-ad49eaca8867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:22.044638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea9f61-ed76-45c4-a00a-ad49eaca8867 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:22.118366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1224/2454] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:59:22 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:59:22.123153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aea7e-1000-45f6-91ad-fc73a905d48a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:22.135837 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aea7e-1000-45f6-91ad-fc73a905d48a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:22.203551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0e5f-1b93-44f0-97c1-ae8e01c43f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:22.204250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0e5f-1b93-44f0-97c1-ae8e01c43f69 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:22.204588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0e5f-1b93-44f0-97c1-ae8e01c43f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:22.204871 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0e5f-1b93-44f0-97c1-ae8e01c43f69 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:22.205642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b0e5f-1b93-44f0-97c1-ae8e01c43f69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:22.207639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b0e5f-1b93-44f0-97c1-ae8e01c43f69 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:22.279205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1231/2455] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:59:22 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:59:22.284203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea9f61-ed76-45c4-a00a-ad49eaca8867 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:22.300430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea9f61-ed76-45c4-a00a-ad49eaca8867 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:22.306711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be51587-4a5f-40e5-9c86-4e4e11777947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:22.308135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be51587-4a5f-40e5-9c86-4e4e11777947 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:22.308665 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be51587-4a5f-40e5-9c86-4e4e11777947 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:22.308868 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be51587-4a5f-40e5-9c86-4e4e11777947 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:22.311550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be51587-4a5f-40e5-9c86-4e4e11777947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:22.311550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be51587-4a5f-40e5-9c86-4e4e11777947 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:22.331275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1225/2456] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:59:22.335431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0e5f-1b93-44f0-97c1-ae8e01c43f69 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:22.347764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0e5f-1b93-44f0-97c1-ae8e01c43f69 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:22.353466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c7410-7cee-437e-9334-8209331ab8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:22.354730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c7410-7cee-437e-9334-8209331ab8cf admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:22.355046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c7410-7cee-437e-9334-8209331ab8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:22.355343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c7410-7cee-437e-9334-8209331ab8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:22.356104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c7410-7cee-437e-9334-8209331ab8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:22.357768 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c7410-7cee-437e-9334-8209331ab8cf admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:22.370371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1232/2457] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:59:22.374689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be51587-4a5f-40e5-9c86-4e4e11777947 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:22.389213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be51587-4a5f-40e5-9c86-4e4e11777947 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:22.395996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afdd62-6e3d-4b91-bca8-7353bca0606f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:22.397590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afdd62-6e3d-4b91-bca8-7353bca0606f admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:22.397869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afdd62-6e3d-4b91-bca8-7353bca0606f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:22.398177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afdd62-6e3d-4b91-bca8-7353bca0606f admin admin] PATH_INFO: `/v3/projects/8331debc88ed4f19b650426e1901b063/users/7e03ea067e27456590c0865766830d73/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:22.421175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afdd62-6e3d-4b91-bca8-7353bca0606f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8331debc88ed4f19b650426e1901b063, user_id=7e03ea067e27456590c0865766830d73, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:22.423923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afdd62-6e3d-4b91-bca8-7353bca0606f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:22.498479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1226/2458] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:59:22 2022] PUT /identity/v3/projects/8331debc88ed4f19b650426e1901b063/users/7e03ea067e27456590c0865766830d73/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:59:22.507608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20c52e-c261-4fd2-83ff-78c250aa6849 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:22.507608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20c52e-c261-4fd2-83ff-78c250aa6849 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:22.507608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20c52e-c261-4fd2-83ff-78c250aa6849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:22.523719 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-fc20c52e-c261-4fd2-83ff-78c250aa6849 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:22.525151 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-fc20c52e-c261-4fd2-83ff-78c250aa6849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:22.593900 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1233/2459] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:22.604409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afdd62-6e3d-4b91-bca8-7353bca0606f admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:22.606302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-42afdd62-6e3d-4b91-bca8-7353bca0606f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:22.637113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:22.691233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afdd62-6e3d-4b91-bca8-7353bca0606f admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:22.697974 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bce714-5c4f-4b2b-a1e6-d0e6ef01f829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:22.699077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bce714-5c4f-4b2b-a1e6-d0e6ef01f829 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:22.699689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bce714-5c4f-4b2b-a1e6-d0e6ef01f829 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:22.700294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bce714-5c4f-4b2b-a1e6-d0e6ef01f829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:22.718619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bce714-5c4f-4b2b-a1e6-d0e6ef01f829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:22.720689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bce714-5c4f-4b2b-a1e6-d0e6ef01f829 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:22.749626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1227/2460] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:59:27.974461 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1e98d-dae1-4db7-8c91-7492e5c8d67b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:27.975488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1e98d-dae1-4db7-8c91-7492e5c8d67b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:27.975923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1e98d-dae1-4db7-8c91-7492e5c8d67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:27.996576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-30c1e98d-dae1-4db7-8c91-7492e5c8d67b None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:27.998102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-30c1e98d-dae1-4db7-8c91-7492e5c8d67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:28.027983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:28.075754 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1234/2461] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 23 15:59:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:28.090830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bce714-5c4f-4b2b-a1e6-d0e6ef01f829 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:28.092352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-50bce714-5c4f-4b2b-a1e6-d0e6ef01f829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:28.191978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bce714-5c4f-4b2b-a1e6-d0e6ef01f829 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:28.198740 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b25e7-8c2a-4e51-8ddd-2742901f50ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:28.199529 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b25e7-8c2a-4e51-8ddd-2742901f50ae service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:28.199913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b25e7-8c2a-4e51-8ddd-2742901f50ae service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:28.200290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b25e7-8c2a-4e51-8ddd-2742901f50ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:28.209958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b25e7-8c2a-4e51-8ddd-2742901f50ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:28.211544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b25e7-8c2a-4e51-8ddd-2742901f50ae service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:28.220604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1228/2462] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 15:59:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:59:29.016393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510253d-58d1-4b2d-ad2b-295bca8485df None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:29.017357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510253d-58d1-4b2d-ad2b-295bca8485df None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:29.017681 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510253d-58d1-4b2d-ad2b-295bca8485df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:29.048068 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-6510253d-58d1-4b2d-ad2b-295bca8485df None None] MFA Rules not processed for user `7e03ea067e27456590c0865766830d73`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:29.053773 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6510253d-58d1-4b2d-ad2b-295bca8485df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:29.125810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1235/2463] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:29.130235 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b25e7-8c2a-4e51-8ddd-2742901f50ae service swift] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:29.142236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b25e7-8c2a-4e51-8ddd-2742901f50ae service swift] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:29.148660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0002bd9-117e-4f50-910a-09c3bc0bafb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:29.149346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0002bd9-117e-4f50-910a-09c3bc0bafb6 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:29.149662 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0002bd9-117e-4f50-910a-09c3bc0bafb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:29.149945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0002bd9-117e-4f50-910a-09c3bc0bafb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:29.150661 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0002bd9-117e-4f50-910a-09c3bc0bafb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:29.152635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0002bd9-117e-4f50-910a-09c3bc0bafb6 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:29.249108 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1229/2464] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:59:29 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:59:29.253178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510253d-58d1-4b2d-ad2b-295bca8485df None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:29.264409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510253d-58d1-4b2d-ad2b-295bca8485df None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:29.330890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2d7ae-8a8e-4125-a94e-cd2e1bb8b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:29.331580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2d7ae-8a8e-4125-a94e-cd2e1bb8b7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:29.331863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2d7ae-8a8e-4125-a94e-cd2e1bb8b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:29.332143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2d7ae-8a8e-4125-a94e-cd2e1bb8b7d9 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:29.332897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2d7ae-8a8e-4125-a94e-cd2e1bb8b7d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:29.334835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2d7ae-8a8e-4125-a94e-cd2e1bb8b7d9 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:29.416936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1236/2465] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:59:29 2022] POST /identity/v3/users => generated 296 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:59:29.421133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0002bd9-117e-4f50-910a-09c3bc0bafb6 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:29.431924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0002bd9-117e-4f50-910a-09c3bc0bafb6 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:29.436518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8be75-9f1c-428a-abbf-e59cc9237797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:29.437214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8be75-9f1c-428a-abbf-e59cc9237797 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:29.437504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8be75-9f1c-428a-abbf-e59cc9237797 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:29.437782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8be75-9f1c-428a-abbf-e59cc9237797 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:29.438516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8be75-9f1c-428a-abbf-e59cc9237797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:29.440111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8be75-9f1c-428a-abbf-e59cc9237797 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:29.459494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1230/2466] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:59:29.460332 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2d7ae-8a8e-4125-a94e-cd2e1bb8b7d9 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:29.471544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2d7ae-8a8e-4125-a94e-cd2e1bb8b7d9 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:29.476328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027b51f-1ac3-4c34-8089-102f6f156496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:29.477247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027b51f-1ac3-4c34-8089-102f6f156496 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:29.477555 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027b51f-1ac3-4c34-8089-102f6f156496 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:29.477865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027b51f-1ac3-4c34-8089-102f6f156496 admin admin] PATH_INFO: `/v3/projects/f9069725e2a14eb9a412f50d226f36c7/users/a00b03687ca0447cab94d0015636602a/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:29.497552 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027b51f-1ac3-4c34-8089-102f6f156496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9069725e2a14eb9a412f50d226f36c7, user_id=a00b03687ca0447cab94d0015636602a, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:29.500438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027b51f-1ac3-4c34-8089-102f6f156496 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:29.595093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1237/2467] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:59:29 2022] PUT /identity/v3/projects/f9069725e2a14eb9a412f50d226f36c7/users/a00b03687ca0447cab94d0015636602a/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 15:59:29.607128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8be75-9f1c-428a-abbf-e59cc9237797 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:29.630452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8be75-9f1c-428a-abbf-e59cc9237797 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:29.697792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972355a0-18d0-4594-98b5-dcab918aae13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:29.698466 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972355a0-18d0-4594-98b5-dcab918aae13 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:29.698755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972355a0-18d0-4594-98b5-dcab918aae13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:29.699034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972355a0-18d0-4594-98b5-dcab918aae13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:29.699760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972355a0-18d0-4594-98b5-dcab918aae13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:29.701386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972355a0-18d0-4594-98b5-dcab918aae13 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:29.714075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1231/2468] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:59:29.716794 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027b51f-1ac3-4c34-8089-102f6f156496 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:29.727497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027b51f-1ac3-4c34-8089-102f6f156496 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:29.733403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce9a6-5ac5-4721-991d-d9f7e669463a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:29.734469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce9a6-5ac5-4721-991d-d9f7e669463a admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:29.734897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce9a6-5ac5-4721-991d-d9f7e669463a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:29.735072 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce9a6-5ac5-4721-991d-d9f7e669463a admin admin] PATH_INFO: `/v3/projects/f9069725e2a14eb9a412f50d226f36c7/users/a00b03687ca0447cab94d0015636602a/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:29.737106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cce9a6-5ac5-4721-991d-d9f7e669463a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9069725e2a14eb9a412f50d226f36c7, user_id=a00b03687ca0447cab94d0015636602a, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:29.742413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cce9a6-5ac5-4721-991d-d9f7e669463a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:29.754064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1238/2469] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:59:29 2022] PUT /identity/v3/projects/f9069725e2a14eb9a412f50d226f36c7/users/a00b03687ca0447cab94d0015636602a/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:59:31.554393 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972355a0-18d0-4594-98b5-dcab918aae13 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:31.556189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-972355a0-18d0-4594-98b5-dcab918aae13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:31.570441 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972355a0-18d0-4594-98b5-dcab918aae13 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:31.583348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:31.631972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:31.633042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:31.633421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:31.634301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] PATH_INFO: `/v3/users/c06753284d314b6b9ef28d77c05b452b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:31.635997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06753284d314b6b9ef28d77c05b452b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:31.638241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:31.824381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06753284d314b6b9ef28d77c05b452b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:31.825580 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06753284d314b6b9ef28d77c05b452b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:31.841012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06753284d314b6b9ef28d77c05b452b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:31.841734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06753284d314b6b9ef28d77c05b452b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:31.849815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6805a191-e555-49e8-b367-68755b47667a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06753284d314b6b9ef28d77c05b452b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:31.939626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1232/2470] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 15:59:31 2022] DELETE /identity/v3/users/c06753284d314b6b9ef28d77c05b452b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:59:32.341818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce9a6-5ac5-4721-991d-d9f7e669463a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:32.346403 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-60cce9a6-5ac5-4721-991d-d9f7e669463a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:32.406298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce9a6-5ac5-4721-991d-d9f7e669463a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:32.421186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:32.494835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:32.496117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:32.496241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:32.505803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] PATH_INFO: `/v3/projects/67441f22b9c4455fa7975cca5a417185` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:32.507308 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] RBAC: Authorizing `identity:delete_project(project_id=67441f22b9c4455fa7975cca5a417185)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:32.509953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:32.666434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67441f22b9c4455fa7975cca5a417185', 'request_id': 'req-65ad152c-a298-40de-b8b9-6f2c47dd983d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:32.679605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67441f22b9c4455fa7975cca5a417185', 'request_id': 'req-65ad152c-a298-40de-b8b9-6f2c47dd983d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:32.693293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67441f22b9c4455fa7975cca5a417185', 'request_id': 'req-65ad152c-a298-40de-b8b9-6f2c47dd983d'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:32.788851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] The token cache is being invalidate because project 67441f22b9c4455fa7975cca5a417185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 15:59:32.789175 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:32.790378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-65ad152c-a298-40de-b8b9-6f2c47dd983d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 15:59:32.791995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1239/2471] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 15:59:32 2022] DELETE /identity/v3/projects/67441f22b9c4455fa7975cca5a417185 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:59:32.797347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd124eb9-93cb-4ccb-8b95-2532bdf1ae1b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:32.797823 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd124eb9-93cb-4ccb-8b95-2532bdf1ae1b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:32.798038 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd124eb9-93cb-4ccb-8b95-2532bdf1ae1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:32.830347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-dd124eb9-93cb-4ccb-8b95-2532bdf1ae1b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:32.832123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dd124eb9-93cb-4ccb-8b95-2532bdf1ae1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:32.940602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1233/2472] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:32.948937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320535aa-79da-4c0e-9d8f-39886f27fd52 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:32.949257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320535aa-79da-4c0e-9d8f-39886f27fd52 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:32.949779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320535aa-79da-4c0e-9d8f-39886f27fd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:32.972511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-320535aa-79da-4c0e-9d8f-39886f27fd52 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:32.973857 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-320535aa-79da-4c0e-9d8f-39886f27fd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:32.983566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1240/2473] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:32.987291 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd124eb9-93cb-4ccb-8b95-2532bdf1ae1b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:33.004716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd124eb9-93cb-4ccb-8b95-2532bdf1ae1b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:33.019875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c5684-810d-4f0e-bb7b-9337345e6fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:33.019875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c5684-810d-4f0e-bb7b-9337345e6fcc admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:33.019875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c5684-810d-4f0e-bb7b-9337345e6fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:33.019875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c5684-810d-4f0e-bb7b-9337345e6fcc admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:33.020234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c5684-810d-4f0e-bb7b-9337345e6fcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 15:59:33.020234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c5684-810d-4f0e-bb7b-9337345e6fcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:33.020234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c5684-810d-4f0e-bb7b-9337345e6fcc admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:33.042626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1234/2474] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 15:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 15:59:33.043663 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320535aa-79da-4c0e-9d8f-39886f27fd52 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:33.090285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320535aa-79da-4c0e-9d8f-39886f27fd52 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:33.097165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639edd87-f0a7-4667-9a72-e4aca2df938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:33.098303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639edd87-f0a7-4667-9a72-e4aca2df938c admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:33.098303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639edd87-f0a7-4667-9a72-e4aca2df938c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:33.098666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639edd87-f0a7-4667-9a72-e4aca2df938c admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:33.099371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639edd87-f0a7-4667-9a72-e4aca2df938c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:33.101370 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639edd87-f0a7-4667-9a72-e4aca2df938c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:33.220448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1241/2475] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:59:33 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:59:33.224515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c5684-810d-4f0e-bb7b-9337345e6fcc admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:33.237745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c5684-810d-4f0e-bb7b-9337345e6fcc admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:33.262864 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:33.313708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a8dac-2788-473d-ab65-5b612c1153e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:33.314331 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a8dac-2788-473d-ab65-5b612c1153e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:33.314778 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a8dac-2788-473d-ab65-5b612c1153e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:33.315063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a8dac-2788-473d-ab65-5b612c1153e6 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:33.315772 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a8dac-2788-473d-ab65-5b612c1153e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:33.317780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a8dac-2788-473d-ab65-5b612c1153e6 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:33.423170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1235/2476] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:59:33 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:59:33.428639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639edd87-f0a7-4667-9a72-e4aca2df938c admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:33.441749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639edd87-f0a7-4667-9a72-e4aca2df938c admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:33.452865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1277123-25a2-4fba-ba00-018764970534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:33.452865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1277123-25a2-4fba-ba00-018764970534 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:33.452865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1277123-25a2-4fba-ba00-018764970534 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:33.452865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1277123-25a2-4fba-ba00-018764970534 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:33.453264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1277123-25a2-4fba-ba00-018764970534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:33.454924 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1277123-25a2-4fba-ba00-018764970534 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:33.473651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1242/2477] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:59:33.477913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a8dac-2788-473d-ab65-5b612c1153e6 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:33.499655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a8dac-2788-473d-ab65-5b612c1153e6 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:33.510707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d6ee7-aaf9-4243-bbf6-192e69d72ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:33.511533 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d6ee7-aaf9-4243-bbf6-192e69d72ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:33.511919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d6ee7-aaf9-4243-bbf6-192e69d72ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:33.514349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d6ee7-aaf9-4243-bbf6-192e69d72ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:33.515219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d6ee7-aaf9-4243-bbf6-192e69d72ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:33.516984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d6ee7-aaf9-4243-bbf6-192e69d72ce7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:33.535446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1236/2478] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:59:33.541431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1277123-25a2-4fba-ba00-018764970534 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:33.568807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1277123-25a2-4fba-ba00-018764970534 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:33.574880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff514f-6023-4329-96de-5b731ae45ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:33.577361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff514f-6023-4329-96de-5b731ae45ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:33.577668 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff514f-6023-4329-96de-5b731ae45ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:33.577858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff514f-6023-4329-96de-5b731ae45ca5 admin admin] PATH_INFO: `/v3/projects/29fc69c9c1c04a64ac1578af40ffe9b6/users/3fc4770081614979bcee97c01d651349/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:33.604450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff514f-6023-4329-96de-5b731ae45ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29fc69c9c1c04a64ac1578af40ffe9b6, user_id=3fc4770081614979bcee97c01d651349, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:33.608309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff514f-6023-4329-96de-5b731ae45ca5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:33.704655 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1243/2479] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:59:33 2022] PUT /identity/v3/projects/29fc69c9c1c04a64ac1578af40ffe9b6/users/3fc4770081614979bcee97c01d651349/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:59:33.709629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7cd66-2852-4422-a950-01eab8466f50 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:33.709868 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7cd66-2852-4422-a950-01eab8466f50 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:33.710146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7cd66-2852-4422-a950-01eab8466f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:33.734677 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-53f7cd66-2852-4422-a950-01eab8466f50 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:33.738867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-53f7cd66-2852-4422-a950-01eab8466f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:33.819851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1237/2480] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:33.829596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff514f-6023-4329-96de-5b731ae45ca5 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:33.831811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2eff514f-6023-4329-96de-5b731ae45ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:33.903172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:33.932734 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff514f-6023-4329-96de-5b731ae45ca5 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:33.936844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7297e-29d8-496b-842a-544aae522f37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:33.937562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7297e-29d8-496b-842a-544aae522f37 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:33.937850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7297e-29d8-496b-842a-544aae522f37 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:33.938266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7297e-29d8-496b-842a-544aae522f37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:33.949258 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7297e-29d8-496b-842a-544aae522f37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:33.950835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7297e-29d8-496b-842a-544aae522f37 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:33.962146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1244/2481] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:59:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:59:36.263927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb704e1-709d-437c-910b-0c118b3aafc7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:36.264899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb704e1-709d-437c-910b-0c118b3aafc7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:36.265344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb704e1-709d-437c-910b-0c118b3aafc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:36.306295 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-dfb704e1-709d-437c-910b-0c118b3aafc7 None None] MFA Rules not processed for user `a00b03687ca0447cab94d0015636602a`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:36.307509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dfb704e1-709d-437c-910b-0c118b3aafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:36.410288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1238/2482] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:36.425996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7297e-29d8-496b-842a-544aae522f37 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:36.428186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ebd7297e-29d8-496b-842a-544aae522f37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:36.590771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7297e-29d8-496b-842a-544aae522f37 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:36.596876 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec79f01-589a-43b8-a868-2c4927b75a3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:36.597566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec79f01-589a-43b8-a868-2c4927b75a3f service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:36.597909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec79f01-589a-43b8-a868-2c4927b75a3f service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:36.598128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec79f01-589a-43b8-a868-2c4927b75a3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:36.606288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3ec79f01-589a-43b8-a868-2c4927b75a3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:36.699855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec79f01-589a-43b8-a868-2c4927b75a3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:36.701555 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec79f01-589a-43b8-a868-2c4927b75a3f service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:36.711993 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1245/2483] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:59:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:59:40.877421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b319a-b346-4b06-87c2-b93be32587d0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:40.878003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b319a-b346-4b06-87c2-b93be32587d0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:40.878102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b319a-b346-4b06-87c2-b93be32587d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:40.913942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-635b319a-b346-4b06-87c2-b93be32587d0 None None] MFA Rules not processed for user `3fc4770081614979bcee97c01d651349`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:40.915718 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-635b319a-b346-4b06-87c2-b93be32587d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:40.933281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:41.012585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1239/2484] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:41.018219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec79f01-589a-43b8-a868-2c4927b75a3f service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:41.101763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec79f01-589a-43b8-a868-2c4927b75a3f service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:41.101763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a12b6-093a-4831-8cf2-7a2f81a218e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:41.101763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a12b6-093a-4831-8cf2-7a2f81a218e3 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:41.101763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a12b6-093a-4831-8cf2-7a2f81a218e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:41.103596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a12b6-093a-4831-8cf2-7a2f81a218e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:41.103596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a12b6-093a-4831-8cf2-7a2f81a218e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:41.103596 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a12b6-093a-4831-8cf2-7a2f81a218e3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:41.342528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1246/2485] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 15:59:41 2022] POST /identity/v3/projects => generated 385 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 15:59:41.347242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b319a-b346-4b06-87c2-b93be32587d0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:41.365707 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b319a-b346-4b06-87c2-b93be32587d0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:41.463667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3a9e8-6422-4914-9f90-8ce2582cc92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:41.463667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3a9e8-6422-4914-9f90-8ce2582cc92b admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:41.463942 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3a9e8-6422-4914-9f90-8ce2582cc92b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:41.464021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e3a9e8-6422-4914-9f90-8ce2582cc92b admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:41.465852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3a9e8-6422-4914-9f90-8ce2582cc92b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:41.467924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e3a9e8-6422-4914-9f90-8ce2582cc92b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:41.588913 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1240/2486] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 15:59:41 2022] POST /identity/v3/users => generated 299 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 15:59:41.590399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a12b6-093a-4831-8cf2-7a2f81a218e3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:41.608533 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a12b6-093a-4831-8cf2-7a2f81a218e3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:41.621966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c1096-653a-465b-bdd1-a9f764d6f2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:41.622687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c1096-653a-465b-bdd1-a9f764d6f2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:41.622987 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c1096-653a-465b-bdd1-a9f764d6f2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:41.623255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c1096-653a-465b-bdd1-a9f764d6f2e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:41.623993 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c1096-653a-465b-bdd1-a9f764d6f2e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:41.626567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c1096-653a-465b-bdd1-a9f764d6f2e8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:41.653976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1247/2487] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 15:59:41.658348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3a9e8-6422-4914-9f90-8ce2582cc92b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:41.670930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e3a9e8-6422-4914-9f90-8ce2582cc92b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:41.676684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264bcc1-fbf6-4970-9111-c86078e4c7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:41.677870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bcc1-fbf6-4970-9111-c86078e4c7f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:41.678165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bcc1-fbf6-4970-9111-c86078e4c7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:41.678448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264bcc1-fbf6-4970-9111-c86078e4c7f3 admin admin] PATH_INFO: `/v3/projects/e8f61d64d47e4978a12339c665c6a4f6/users/35a517ad6e5442b7a673d1caf1dd2d7a/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:41.702045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264bcc1-fbf6-4970-9111-c86078e4c7f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f61d64d47e4978a12339c665c6a4f6, user_id=35a517ad6e5442b7a673d1caf1dd2d7a, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:41.707953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264bcc1-fbf6-4970-9111-c86078e4c7f3 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:41.811455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1241/2488] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:59:41 2022] PUT /identity/v3/projects/e8f61d64d47e4978a12339c665c6a4f6/users/35a517ad6e5442b7a673d1caf1dd2d7a/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:59:41.815459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c1096-653a-465b-bdd1-a9f764d6f2e8 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:41.826174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c1096-653a-465b-bdd1-a9f764d6f2e8 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:41.852791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:41.883352 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415e1a9-8f46-441d-a8ef-b98d89871f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:41.884279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415e1a9-8f46-441d-a8ef-b98d89871f37 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:41.884588 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415e1a9-8f46-441d-a8ef-b98d89871f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:41.884861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415e1a9-8f46-441d-a8ef-b98d89871f37 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:41.885582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415e1a9-8f46-441d-a8ef-b98d89871f37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:41.887170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415e1a9-8f46-441d-a8ef-b98d89871f37 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:41.900694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1248/2489] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 15:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 15:59:41.902136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264bcc1-fbf6-4970-9111-c86078e4c7f3 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:41.917258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264bcc1-fbf6-4970-9111-c86078e4c7f3 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:41.923243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be270f8-8197-4fbc-8718-fe557c6cd90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:41.924134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be270f8-8197-4fbc-8718-fe557c6cd90e admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:41.924438 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be270f8-8197-4fbc-8718-fe557c6cd90e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:41.924739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be270f8-8197-4fbc-8718-fe557c6cd90e admin admin] PATH_INFO: `/v3/projects/e8f61d64d47e4978a12339c665c6a4f6/users/35a517ad6e5442b7a673d1caf1dd2d7a/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:41.926740 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be270f8-8197-4fbc-8718-fe557c6cd90e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f61d64d47e4978a12339c665c6a4f6, user_id=35a517ad6e5442b7a673d1caf1dd2d7a, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:41.929457 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be270f8-8197-4fbc-8718-fe557c6cd90e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:41.939493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1242/2490] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 15:59:41 2022] PUT /identity/v3/projects/e8f61d64d47e4978a12339c665c6a4f6/users/35a517ad6e5442b7a673d1caf1dd2d7a/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 15:59:49.065976 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda035c6-e40a-4127-b0e9-b6ac4f695391 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:49.066714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda035c6-e40a-4127-b0e9-b6ac4f695391 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:49.066873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda035c6-e40a-4127-b0e9-b6ac4f695391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:49.108486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-fda035c6-e40a-4127-b0e9-b6ac4f695391 None None] MFA Rules not processed for user `35a517ad6e5442b7a673d1caf1dd2d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:49.109879 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-fda035c6-e40a-4127-b0e9-b6ac4f695391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:49.211163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1249/2491] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 15:59:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:49.223650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be270f8-8197-4fbc-8718-fe557c6cd90e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:49.227960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3be270f8-8197-4fbc-8718-fe557c6cd90e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:49.311139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:49.313848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be270f8-8197-4fbc-8718-fe557c6cd90e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:49.317914 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6662b64-9847-441f-99c0-fe5839f7ff25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '142e6e44d6544f6a891ed3f913de74ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:49.318574 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662b64-9847-441f-99c0-fe5839f7ff25 service glance] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:49.318860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662b64-9847-441f-99c0-fe5839f7ff25 service glance] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:49.319140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662b64-9847-441f-99c0-fe5839f7ff25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:49.380463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6662b64-9847-441f-99c0-fe5839f7ff25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:49.382057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6662b64-9847-441f-99c0-fe5839f7ff25 service glance] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:49.394051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1243/2492] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 15:59:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 15:59:49.616484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f36baa-aa7e-4d1a-baa8-345fe0a7724f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:49.617448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f36baa-aa7e-4d1a-baa8-345fe0a7724f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:49.617828 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f36baa-aa7e-4d1a-baa8-345fe0a7724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:49.638424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-f5f36baa-aa7e-4d1a-baa8-345fe0a7724f None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 15:59:49.640053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f5f36baa-aa7e-4d1a-baa8-345fe0a7724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:49.708288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1250/2493] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 15:59:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 15:59:49.723239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6662b64-9847-441f-99c0-fe5839f7ff25 service glance] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 15:59:49.724721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c6662b64-9847-441f-99c0-fe5839f7ff25 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 15:59:49.827330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6662b64-9847-441f-99c0-fe5839f7ff25 service glance] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 15:59:49.832491 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5410e-1bec-416b-a9cc-36e7ca85f691 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 15:59:49.833152 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5410e-1bec-416b-a9cc-36e7ca85f691 service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 15:59:49.833436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5410e-1bec-416b-a9cc-36e7ca85f691 service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 15:59:49.843679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5410e-1bec-416b-a9cc-36e7ca85f691 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 15:59:49.843679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5410e-1bec-416b-a9cc-36e7ca85f691 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 15:59:49.843928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5410e-1bec-416b-a9cc-36e7ca85f691 service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 15:59:49.855652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1244/2494] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 15:59:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 15:59:59.695817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=75 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 15:59:59.851045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=58 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:00:00.748575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500526c-4a2f-4f66-978b-880fded5e2f6 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:00:00.749122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500526c-4a2f-4f66-978b-880fded5e2f6 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:00:00.749122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500526c-4a2f-4f66-978b-880fded5e2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:00:00.766914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a500526c-4a2f-4f66-978b-880fded5e2f6 None None] MFA Rules not processed for user `3fc4770081614979bcee97c01d651349`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:00:00.768112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a500526c-4a2f-4f66-978b-880fded5e2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:00:00.775086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1251/2495] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:00:01.543981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c747e-f6e6-41e2-81fb-9a1a77fba6f8 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:00:01.544547 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c747e-f6e6-41e2-81fb-9a1a77fba6f8 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:00:01.544652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c747e-f6e6-41e2-81fb-9a1a77fba6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:00:01.562467 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-bf5c747e-f6e6-41e2-81fb-9a1a77fba6f8 None None] MFA Rules not processed for user `7e03ea067e27456590c0865766830d73`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:00:01.563600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-bf5c747e-f6e6-41e2-81fb-9a1a77fba6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:00:01.634033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1245/2496] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:00:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:00:04.690005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79402d7b-976c-4bde-b00c-016afddb301d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:00:04.690448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79402d7b-976c-4bde-b00c-016afddb301d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:00:04.690448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79402d7b-976c-4bde-b00c-016afddb301d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:00:04.709799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-79402d7b-976c-4bde-b00c-016afddb301d None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:00:04.711105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-79402d7b-976c-4bde-b00c-016afddb301d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:00:04.718657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1252/2497] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 16:00:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:00:04.734753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c747e-f6e6-41e2-81fb-9a1a77fba6f8 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:00:04.748461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c747e-f6e6-41e2-81fb-9a1a77fba6f8 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:00:04.753872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fcef3-b5e6-4743-bfff-9250b4e4b92e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:00:04.754736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fcef3-b5e6-4743-bfff-9250b4e4b92e service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:00:04.755024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fcef3-b5e6-4743-bfff-9250b4e4b92e service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:00:04.755302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fcef3-b5e6-4743-bfff-9250b4e4b92e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:00:04.767366 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fcef3-b5e6-4743-bfff-9250b4e4b92e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:00:04.769147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fcef3-b5e6-4743-bfff-9250b4e4b92e service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:00:04.778239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1246/2498] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 16:00:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 16:00:14.703981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=6 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:00:14.774711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=31 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:00:21.794503 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc751-0924-4857-9307-adc762623daa None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:00:21.795929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc751-0924-4857-9307-adc762623daa None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:00:21.795929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc751-0924-4857-9307-adc762623daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:00:21.814327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-118bc751-0924-4857-9307-adc762623daa None None] MFA Rules not processed for user `5b52924339de4cc08fe35dbf2a5fdcd0`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:00:21.815642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-118bc751-0924-4857-9307-adc762623daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:00:21.873634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1253/2499] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:00:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:00:30.340944 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ac293-cead-45e2-bfe8-0c47092daf5e None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:00:30.341515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ac293-cead-45e2-bfe8-0c47092daf5e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:00:30.341515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ac293-cead-45e2-bfe8-0c47092daf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:00:30.359389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-0e5ac293-cead-45e2-bfe8-0c47092daf5e None None] MFA Rules not processed for user `36e2bae836f5409f822af1f2919e4b80`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:00:30.360622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0e5ac293-cead-45e2-bfe8-0c47092daf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:00:30.417618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1247/2500] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:00:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:00:31.866811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=25 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:00:39.799255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc751-0924-4857-9307-adc762623daa None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:00:39.800804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-118bc751-0924-4857-9307-adc762623daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:00:39.862296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc751-0924-4857-9307-adc762623daa None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:00:39.867468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6199355-5736-400e-ad02-b9d352301a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:00:39.867468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6199355-5736-400e-ad02-b9d352301a99 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:00:39.867468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6199355-5736-400e-ad02-b9d352301a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:00:39.867468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6199355-5736-400e-ad02-b9d352301a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:00:39.928160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6199355-5736-400e-ad02-b9d352301a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:00:39.929646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6199355-5736-400e-ad02-b9d352301a99 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:00:39.945667 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1254/2501] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:00:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:00:40.408186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=25 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:00:46.765179 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ac293-cead-45e2-bfe8-0c47092daf5e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:00:46.775269 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ac293-cead-45e2-bfe8-0c47092daf5e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:00:46.779052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd35bcf-9be8-45c9-b5be-fe1f825eba9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:00:46.779715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd35bcf-9be8-45c9-b5be-fe1f825eba9c service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:00:46.780003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd35bcf-9be8-45c9-b5be-fe1f825eba9c service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:00:46.780280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd35bcf-9be8-45c9-b5be-fe1f825eba9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:00:46.790723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd35bcf-9be8-45c9-b5be-fe1f825eba9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:00:46.793626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd35bcf-9be8-45c9-b5be-fe1f825eba9c service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:00:46.802941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1248/2502] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:00:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:00:49.935580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=50 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:00:56.797943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=6 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:06.093080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6199355-5736-400e-ad02-b9d352301a99 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:06.094526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b6199355-5736-400e-ad02-b9d352301a99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:06.164972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6199355-5736-400e-ad02-b9d352301a99 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:06.169536 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969eaa73-cebe-49e1-87bf-d967f3b23f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:06.170206 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969eaa73-cebe-49e1-87bf-d967f3b23f41 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:06.170490 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969eaa73-cebe-49e1-87bf-d967f3b23f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:06.175184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969eaa73-cebe-49e1-87bf-d967f3b23f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:06.175184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-969eaa73-cebe-49e1-87bf-d967f3b23f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:06.251762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969eaa73-cebe-49e1-87bf-d967f3b23f41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:06.253291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969eaa73-cebe-49e1-87bf-d967f3b23f41 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:06.264480 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1255/2503] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:01:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:01:08.160672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5f494-7d5d-4153-9d2c-4b90afb0ae9b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:08.161197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5f494-7d5d-4153-9d2c-4b90afb0ae9b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:08.161293 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5f494-7d5d-4153-9d2c-4b90afb0ae9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:08.179367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-5fa5f494-7d5d-4153-9d2c-4b90afb0ae9b None None] MFA Rules not processed for user `83bb340d10e44bedad5fc9ddaf332552`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:01:08.180741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5fa5f494-7d5d-4153-9d2c-4b90afb0ae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:08.234555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1249/2504] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:01:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:01:08.270769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969eaa73-cebe-49e1-87bf-d967f3b23f41 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:08.272246 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-969eaa73-cebe-49e1-87bf-d967f3b23f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:08.284555 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969eaa73-cebe-49e1-87bf-d967f3b23f41 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:08.288460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1580454-1ea5-46c4-addf-aee08c7bf92c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '142e6e44d6544f6a891ed3f913de74ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:08.290602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1580454-1ea5-46c4-addf-aee08c7bf92c service glance] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:08.290602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1580454-1ea5-46c4-addf-aee08c7bf92c service glance] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:08.290602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1580454-1ea5-46c4-addf-aee08c7bf92c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:08.291712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e1580454-1ea5-46c4-addf-aee08c7bf92c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:08.306605 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1580454-1ea5-46c4-addf-aee08c7bf92c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:08.308103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1580454-1ea5-46c4-addf-aee08c7bf92c service glance] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:08.319080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1256/2505] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:01:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:01:16.872814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5f494-7d5d-4153-9d2c-4b90afb0ae9b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:16.875323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5fa5f494-7d5d-4153-9d2c-4b90afb0ae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:16.891194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5f494-7d5d-4153-9d2c-4b90afb0ae9b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:16.897214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76262802-ccf5-4ce8-b490-fc81eb5a5eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:16.897857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76262802-ccf5-4ce8-b490-fc81eb5a5eb4 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:16.898135 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76262802-ccf5-4ce8-b490-fc81eb5a5eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:16.898414 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76262802-ccf5-4ce8-b490-fc81eb5a5eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:16.900137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-76262802-ccf5-4ce8-b490-fc81eb5a5eb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:16.910902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76262802-ccf5-4ce8-b490-fc81eb5a5eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:16.913511 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76262802-ccf5-4ce8-b490-fc81eb5a5eb4 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:16.922030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1250/2506] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:01:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:01:18.314172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=60 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:20.957300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1580454-1ea5-46c4-addf-aee08c7bf92c service glance] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:20.958927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e1580454-1ea5-46c4-addf-aee08c7bf92c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:20.970727 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1580454-1ea5-46c4-addf-aee08c7bf92c service glance] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:20.974427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b168876-193b-473a-950b-d44165d70ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:20.975015 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b168876-193b-473a-950b-d44165d70ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:20.975304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b168876-193b-473a-950b-d44165d70ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:20.975582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b168876-193b-473a-950b-d44165d70ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:20.977377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5b168876-193b-473a-950b-d44165d70ea8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:20.988865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b168876-193b-473a-950b-d44165d70ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:20.990291 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b168876-193b-473a-950b-d44165d70ea8 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:20.999459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1257/2507] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:01:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:01:22.281524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76262802-ccf5-4ce8-b490-fc81eb5a5eb4 service nova] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:22.293504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76262802-ccf5-4ce8-b490-fc81eb5a5eb4 service nova] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:22.297327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d2b0f-e410-4589-b764-d155d71a5d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:22.302629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2b0f-e410-4589-b764-d155d71a5d31 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:22.302629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2b0f-e410-4589-b764-d155d71a5d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:22.302629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2b0f-e410-4589-b764-d155d71a5d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:22.302629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d79d2b0f-e410-4589-b764-d155d71a5d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:22.310098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d2b0f-e410-4589-b764-d155d71a5d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:22.311514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d2b0f-e410-4589-b764-d155d71a5d31 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:22.320134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1251/2508] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:01:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:01:30.995057 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=6 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:31.376193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b168876-193b-473a-950b-d44165d70ea8 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:31.389410 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b168876-193b-473a-950b-d44165d70ea8 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:31.395345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a4cc3-a681-437f-966f-6e5ab2b6c68c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:31.396003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a4cc3-a681-437f-966f-6e5ab2b6c68c service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:31.396323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a4cc3-a681-437f-966f-6e5ab2b6c68c service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:31.396669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a4cc3-a681-437f-966f-6e5ab2b6c68c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:31.398449 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f52a4cc3-a681-437f-966f-6e5ab2b6c68c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:31.412964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a4cc3-a681-437f-966f-6e5ab2b6c68c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:31.414620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a4cc3-a681-437f-966f-6e5ab2b6c68c service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:31.427816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1258/2509] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:01:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:01:32.317531 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=37 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:38.318307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d2b0f-e410-4589-b764-d155d71a5d31 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:38.319836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d79d2b0f-e410-4589-b764-d155d71a5d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:38.386361 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d2b0f-e410-4589-b764-d155d71a5d31 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:38.390922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:38.391808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:38.392073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:38.392345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] PATH_INFO: `/v3/users/36e2bae836f5409f822af1f2919e4b80` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:38.393563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e2bae836f5409f822af1f2919e4b80)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:38.395483 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:38.551528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e2bae836f5409f822af1f2919e4b80'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:38.552173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e2bae836f5409f822af1f2919e4b80'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:38.563925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e2bae836f5409f822af1f2919e4b80'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:38.564521 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e2bae836f5409f822af1f2919e4b80'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:38.571327 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e2bae836f5409f822af1f2919e4b80'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:38.640600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1252/2510] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:01:38 2022] DELETE /identity/v3/users/36e2bae836f5409f822af1f2919e4b80 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:01:38.878575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a4cc3-a681-437f-966f-6e5ab2b6c68c service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:38.889105 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a4cc3-a681-437f-966f-6e5ab2b6c68c service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:38.948281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:38.949217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:38.949497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:38.949774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] PATH_INFO: `/v3/projects/53d22d201cab4638bf0ae9392fd79233` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:38.950934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d22d201cab4638bf0ae9392fd79233)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:38.953010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:39.084216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d22d201cab4638bf0ae9392fd79233', 'request_id': 'req-f1c0c648-e6b5-465a-abe1-57659edfcd9f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:39.090560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d22d201cab4638bf0ae9392fd79233', 'request_id': 'req-f1c0c648-e6b5-465a-abe1-57659edfcd9f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:39.096377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d22d201cab4638bf0ae9392fd79233', 'request_id': 'req-f1c0c648-e6b5-465a-abe1-57659edfcd9f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:39.168499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] The token cache is being invalidate because project 53d22d201cab4638bf0ae9392fd79233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:01:39.170627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:39.170627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:39.171704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1259/2511] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:01:38 2022] DELETE /identity/v3/projects/53d22d201cab4638bf0ae9392fd79233 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:01:46.627029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:46.629225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:46.686811 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1d381-d006-4a62-bb71-ea5e1982fb3c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:46.719252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:46.736215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2f8ee-7f23-413f-aa47-81f022afa173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:46.736916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2f8ee-7f23-413f-aa47-81f022afa173 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:46.737197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2f8ee-7f23-413f-aa47-81f022afa173 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:46.737475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2f8ee-7f23-413f-aa47-81f022afa173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:46.740013 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5de2f8ee-7f23-413f-aa47-81f022afa173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:46.803293 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2f8ee-7f23-413f-aa47-81f022afa173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:46.804696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2f8ee-7f23-413f-aa47-81f022afa173 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:46.814243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1253/2512] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:01:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:01:49.165170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:50.477109 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:50.478671 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:50.555033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0c648-e6b5-465a-abe1-57659edfcd9f admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:50.560051 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9e2fa-07cd-48f5-b8d8-c22bb498d8f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:50.560738 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9e2fa-07cd-48f5-b8d8-c22bb498d8f4 service placement] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:50.561085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9e2fa-07cd-48f5-b8d8-c22bb498d8f4 service placement] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:50.561374 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9e2fa-07cd-48f5-b8d8-c22bb498d8f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:50.563207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a3e9e2fa-07cd-48f5-b8d8-c22bb498d8f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:50.574858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9e2fa-07cd-48f5-b8d8-c22bb498d8f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:50.578576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9e2fa-07cd-48f5-b8d8-c22bb498d8f4 service placement] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:50.586549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1260/2513] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:01:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:01:50.614163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2f8ee-7f23-413f-aa47-81f022afa173 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:50.615962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5de2f8ee-7f23-413f-aa47-81f022afa173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:50.629416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2f8ee-7f23-413f-aa47-81f022afa173 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:50.634708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:50.635595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:50.635879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:50.636152 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] PATH_INFO: `/v3/users/5b52924339de4cc08fe35dbf2a5fdcd0` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:50.637626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b52924339de4cc08fe35dbf2a5fdcd0)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:50.639741 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:50.832021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b52924339de4cc08fe35dbf2a5fdcd0'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:50.832656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b52924339de4cc08fe35dbf2a5fdcd0'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:50.844294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b52924339de4cc08fe35dbf2a5fdcd0'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:50.844955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b52924339de4cc08fe35dbf2a5fdcd0'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:50.852267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-9a808e34-b84c-49e6-9353-6c97f5360994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b52924339de4cc08fe35dbf2a5fdcd0'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:50.910354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1254/2514] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:01:50 2022] DELETE /identity/v3/users/5b52924339de4cc08fe35dbf2a5fdcd0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:01:51.168898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9e2fa-07cd-48f5-b8d8-c22bb498d8f4 service placement] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:51.179953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9e2fa-07cd-48f5-b8d8-c22bb498d8f4 service placement] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:51.237296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:51.238242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:51.238529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:51.238804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] PATH_INFO: `/v3/projects/b795cdc9b51647bf96a3da010d94c9cd` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:51.240025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b795cdc9b51647bf96a3da010d94c9cd)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:51.242077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:51.364002 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b795cdc9b51647bf96a3da010d94c9cd', 'request_id': 'req-894bbdff-eb2c-427f-ac6b-8f37d27af80a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:51.370563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b795cdc9b51647bf96a3da010d94c9cd', 'request_id': 'req-894bbdff-eb2c-427f-ac6b-8f37d27af80a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:51.376181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b795cdc9b51647bf96a3da010d94c9cd', 'request_id': 'req-894bbdff-eb2c-427f-ac6b-8f37d27af80a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:51.429610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] The token cache is being invalidate because project b795cdc9b51647bf96a3da010d94c9cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:01:51.429905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:51.430581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-894bbdff-eb2c-427f-ac6b-8f37d27af80a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:01:51.432147 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1261/2515] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:01:51 2022] DELETE /identity/v3/projects/b795cdc9b51647bf96a3da010d94c9cd => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:01:51.437252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ab181-0ed7-456d-ab3f-c564a3d44b1d None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:51.437541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ab181-0ed7-456d-ab3f-c564a3d44b1d None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:51.437822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ab181-0ed7-456d-ab3f-c564a3d44b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:51.456935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-5a5ab181-0ed7-456d-ab3f-c564a3d44b1d None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:01:51.458133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5a5ab181-0ed7-456d-ab3f-c564a3d44b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:51.508029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:51.517165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1255/2516] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:01:51.525600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b35f8e-3781-4dd1-b2d5-6fc83907b26b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:51.526084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b35f8e-3781-4dd1-b2d5-6fc83907b26b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:51.526330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b35f8e-3781-4dd1-b2d5-6fc83907b26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:51.548283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-98b35f8e-3781-4dd1-b2d5-6fc83907b26b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:01:51.549568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-98b35f8e-3781-4dd1-b2d5-6fc83907b26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:51.557610 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1262/2517] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:01:51.561845 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ab181-0ed7-456d-ab3f-c564a3d44b1d None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:51.571970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ab181-0ed7-456d-ab3f-c564a3d44b1d None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:51.578353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9eef6-dbdc-489f-bbf7-3cbeabc5a9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:51.579022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9eef6-dbdc-489f-bbf7-3cbeabc5a9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:51.579305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9eef6-dbdc-489f-bbf7-3cbeabc5a9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:51.579584 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9eef6-dbdc-489f-bbf7-3cbeabc5a9e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:51.580309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9eef6-dbdc-489f-bbf7-3cbeabc5a9e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:01:51.580656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9eef6-dbdc-489f-bbf7-3cbeabc5a9e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:51.582898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9eef6-dbdc-489f-bbf7-3cbeabc5a9e0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:51.595217 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1256/2518] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:01:51.599597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b35f8e-3781-4dd1-b2d5-6fc83907b26b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:51.609071 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b35f8e-3781-4dd1-b2d5-6fc83907b26b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:51.614634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8bde8-3f47-4040-a097-a0603fc84756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:51.615297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8bde8-3f47-4040-a097-a0603fc84756 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:51.615595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8bde8-3f47-4040-a097-a0603fc84756 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:51.615865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8bde8-3f47-4040-a097-a0603fc84756 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:51.616615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8bde8-3f47-4040-a097-a0603fc84756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:51.618518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8bde8-3f47-4040-a097-a0603fc84756 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:51.629893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:51.693123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1263/2519] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:01:51 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:01:51.697226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9eef6-dbdc-489f-bbf7-3cbeabc5a9e0 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:51.707471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9eef6-dbdc-489f-bbf7-3cbeabc5a9e0 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:51.762904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679cff8-d473-481e-8c77-1ff5412d041b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:51.763581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679cff8-d473-481e-8c77-1ff5412d041b admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:51.763868 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679cff8-d473-481e-8c77-1ff5412d041b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:51.764146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679cff8-d473-481e-8c77-1ff5412d041b admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:51.764867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679cff8-d473-481e-8c77-1ff5412d041b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:51.766816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679cff8-d473-481e-8c77-1ff5412d041b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:51.836009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1257/2520] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:01:51 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:01:51.840222 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8bde8-3f47-4040-a097-a0603fc84756 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:51.850772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8bde8-3f47-4040-a097-a0603fc84756 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:51.855362 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c415120-3f4f-4ac8-aa4f-81f30a5173cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:51.861930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c415120-3f4f-4ac8-aa4f-81f30a5173cc admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:51.861930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c415120-3f4f-4ac8-aa4f-81f30a5173cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:51.861930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c415120-3f4f-4ac8-aa4f-81f30a5173cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:51.861930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c415120-3f4f-4ac8-aa4f-81f30a5173cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:51.861930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c415120-3f4f-4ac8-aa4f-81f30a5173cc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:51.869582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1264/2521] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 16:01:51.873716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679cff8-d473-481e-8c77-1ff5412d041b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:51.885508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679cff8-d473-481e-8c77-1ff5412d041b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:51.890659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7d058-fa97-4236-8b63-83767c55d33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:51.890967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d058-fa97-4236-8b63-83767c55d33c admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:51.891255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d058-fa97-4236-8b63-83767c55d33c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:51.891536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d058-fa97-4236-8b63-83767c55d33c admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:51.892266 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7d058-fa97-4236-8b63-83767c55d33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:51.893927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7d058-fa97-4236-8b63-83767c55d33c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:51.903805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1258/2522] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 16:01:51.907984 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c415120-3f4f-4ac8-aa4f-81f30a5173cc admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:51.917841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c415120-3f4f-4ac8-aa4f-81f30a5173cc admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:51.922548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc5ea4-9005-4e46-af5e-03eae027b689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:51.923416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc5ea4-9005-4e46-af5e-03eae027b689 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:51.923702 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc5ea4-9005-4e46-af5e-03eae027b689 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:51.923977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc5ea4-9005-4e46-af5e-03eae027b689 admin admin] PATH_INFO: `/v3/projects/fc4de09e5a074e53a8eb787794367e74/users/929c01c856004eb68f6eb397e9976309/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:51.939196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc5ea4-9005-4e46-af5e-03eae027b689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4de09e5a074e53a8eb787794367e74, user_id=929c01c856004eb68f6eb397e9976309, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:51.942477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc5ea4-9005-4e46-af5e-03eae027b689 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:52.007328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1265/2523] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:01:51 2022] PUT /identity/v3/projects/fc4de09e5a074e53a8eb787794367e74/users/929c01c856004eb68f6eb397e9976309/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:01:52.012423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42144abd-2a83-4717-ad86-21dda3580a5c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:52.018786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42144abd-2a83-4717-ad86-21dda3580a5c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:52.018786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42144abd-2a83-4717-ad86-21dda3580a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:52.030893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-42144abd-2a83-4717-ad86-21dda3580a5c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:01:52.032100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-42144abd-2a83-4717-ad86-21dda3580a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:52.090645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1259/2524] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:01:52.100813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc5ea4-9005-4e46-af5e-03eae027b689 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:52.102484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-37dc5ea4-9005-4e46-af5e-03eae027b689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:52.159894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc5ea4-9005-4e46-af5e-03eae027b689 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:52.163508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42914596-c0db-4aa0-b6db-fda0b4d0a328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:52.164148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42914596-c0db-4aa0-b6db-fda0b4d0a328 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:52.164460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42914596-c0db-4aa0-b6db-fda0b4d0a328 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:52.164741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42914596-c0db-4aa0-b6db-fda0b4d0a328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:52.173609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42914596-c0db-4aa0-b6db-fda0b4d0a328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:52.175634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42914596-c0db-4aa0-b6db-fda0b4d0a328 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:52.184979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1266/2525] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:01:57.426753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1156a7b-fa62-464d-8ccc-333722a1f433 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:57.427359 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1156a7b-fa62-464d-8ccc-333722a1f433 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:57.427577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1156a7b-fa62-464d-8ccc-333722a1f433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:57.456949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a1156a7b-fa62-464d-8ccc-333722a1f433 None None] MFA Rules not processed for user `929c01c856004eb68f6eb397e9976309`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:01:57.458121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a1156a7b-fa62-464d-8ccc-333722a1f433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:57.518303 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:01:57.529168 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1260/2526] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:01:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:01:57.544084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42914596-c0db-4aa0-b6db-fda0b4d0a328 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:01:57.545837 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-42914596-c0db-4aa0-b6db-fda0b4d0a328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:01:57.641469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42914596-c0db-4aa0-b6db-fda0b4d0a328 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:01:57.647939 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db142937-336d-4fdf-a91d-111000878960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:01:57.648755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db142937-336d-4fdf-a91d-111000878960 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:01:57.649194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db142937-336d-4fdf-a91d-111000878960 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:01:57.649318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db142937-336d-4fdf-a91d-111000878960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:01:57.658058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db142937-336d-4fdf-a91d-111000878960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:01:57.664769 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db142937-336d-4fdf-a91d-111000878960 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:01:57.669397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1267/2527] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:01:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:02:00.039146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6361b0d-24d9-4312-9310-d44c5aa114c2 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:00.039666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6361b0d-24d9-4312-9310-d44c5aa114c2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:00.039785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6361b0d-24d9-4312-9310-d44c5aa114c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:00.060953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e6361b0d-24d9-4312-9310-d44c5aa114c2 None None] MFA Rules not processed for user `929c01c856004eb68f6eb397e9976309`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:02:00.060953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e6361b0d-24d9-4312-9310-d44c5aa114c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:00.073490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1261/2528] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:02:00.421017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db142937-336d-4fdf-a91d-111000878960 service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:00.430873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db142937-336d-4fdf-a91d-111000878960 service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:00.434744 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6789a2-522b-49fd-9906-fc3fb578b396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:00.435430 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6789a2-522b-49fd-9906-fc3fb578b396 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:00.435715 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6789a2-522b-49fd-9906-fc3fb578b396 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:00.435995 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6789a2-522b-49fd-9906-fc3fb578b396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:00.437831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ab6789a2-522b-49fd-9906-fc3fb578b396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:00.449604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6789a2-522b-49fd-9906-fc3fb578b396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:00.451178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6789a2-522b-49fd-9906-fc3fb578b396 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:00.461067 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1268/2529] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:02:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:02:00.863512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6361b0d-24d9-4312-9310-d44c5aa114c2 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:00.865173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e6361b0d-24d9-4312-9310-d44c5aa114c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:00.943831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6361b0d-24d9-4312-9310-d44c5aa114c2 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:00.949330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b725a08-bdb5-45bc-9ff3-ce62a443cca5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:00.949984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b725a08-bdb5-45bc-9ff3-ce62a443cca5 service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:00.950268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b725a08-bdb5-45bc-9ff3-ce62a443cca5 service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:00.950644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b725a08-bdb5-45bc-9ff3-ce62a443cca5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:00.952802 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1b725a08-bdb5-45bc-9ff3-ce62a443cca5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:00.967020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b725a08-bdb5-45bc-9ff3-ce62a443cca5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:00.969376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b725a08-bdb5-45bc-9ff3-ce62a443cca5 service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:00.980884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1262/2530] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:02:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:02:10.273710 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6789a2-522b-49fd-9906-fc3fb578b396 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:10.275237 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ab6789a2-522b-49fd-9906-fc3fb578b396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:10.296651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6789a2-522b-49fd-9906-fc3fb578b396 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:10.302085 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e980380-3409-4c69-ab8c-c4fa84dc176c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:10.305563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e980380-3409-4c69-ab8c-c4fa84dc176c service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:10.305563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e980380-3409-4c69-ab8c-c4fa84dc176c service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:10.305563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e980380-3409-4c69-ab8c-c4fa84dc176c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:10.305563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3e980380-3409-4c69-ab8c-c4fa84dc176c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:10.318207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e980380-3409-4c69-ab8c-c4fa84dc176c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:10.319885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e980380-3409-4c69-ab8c-c4fa84dc176c service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:10.333453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1269/2531] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:02:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:02:10.975948 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=38 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:20.327592 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:31.837268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c980e-dbb0-407e-903b-9840264c33a0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:31.837718 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c980e-dbb0-407e-903b-9840264c33a0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:31.837827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c980e-dbb0-407e-903b-9840264c33a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:31.855917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-bc9c980e-dbb0-407e-903b-9840264c33a0 None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:02:31.857138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-bc9c980e-dbb0-407e-903b-9840264c33a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:31.914033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1263/2532] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 23 16:02:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:02:31.953667 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e980380-3409-4c69-ab8c-c4fa84dc176c service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:31.955077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3e980380-3409-4c69-ab8c-c4fa84dc176c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:32.023909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e980380-3409-4c69-ab8c-c4fa84dc176c service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:32.028941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d463429-b56f-4aed-9c46-edc9ba26cac7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:32.030101 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d463429-b56f-4aed-9c46-edc9ba26cac7 service swift] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:32.030387 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d463429-b56f-4aed-9c46-edc9ba26cac7 service swift] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:32.030666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d463429-b56f-4aed-9c46-edc9ba26cac7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:32.039307 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d463429-b56f-4aed-9c46-edc9ba26cac7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:32.040926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d463429-b56f-4aed-9c46-edc9ba26cac7 service swift] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:32.049524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1270/2533] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 16:02:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 16:02:36.810249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c980e-dbb0-407e-903b-9840264c33a0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:36.812849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-bc9c980e-dbb0-407e-903b-9840264c33a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:36.828202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c980e-dbb0-407e-903b-9840264c33a0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:36.833417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e53fc-f7fb-4076-8251-847688c78717 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:36.834080 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e53fc-f7fb-4076-8251-847688c78717 service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:36.834365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e53fc-f7fb-4076-8251-847688c78717 service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:36.834644 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e53fc-f7fb-4076-8251-847688c78717 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:36.836456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f31e53fc-f7fb-4076-8251-847688c78717 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:36.849221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e53fc-f7fb-4076-8251-847688c78717 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:36.850816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e53fc-f7fb-4076-8251-847688c78717 service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:36.862277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1264/2534] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:02:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:02:40.770158 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d463429-b56f-4aed-9c46-edc9ba26cac7 service swift] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:40.771625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8d463429-b56f-4aed-9c46-edc9ba26cac7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:40.786103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d463429-b56f-4aed-9c46-edc9ba26cac7 service swift] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:40.790690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:40.791566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:40.791854 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:40.792163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] PATH_INFO: `/v3/users/3fc4770081614979bcee97c01d651349` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:40.793379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc4770081614979bcee97c01d651349)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:40.795309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:40.981048 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc4770081614979bcee97c01d651349'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:40.981666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc4770081614979bcee97c01d651349'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:40.982283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc4770081614979bcee97c01d651349'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:40.989148 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc4770081614979bcee97c01d651349'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:41.001936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc4770081614979bcee97c01d651349'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:41.083914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1271/2535] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:02:40 2022] DELETE /identity/v3/users/3fc4770081614979bcee97c01d651349 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:02:41.310621 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e53fc-f7fb-4076-8251-847688c78717 service placement] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:41.320792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e53fc-f7fb-4076-8251-847688c78717 service placement] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:41.340382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:41.341847 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:41.378701 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd2c3a-1076-46e5-8cf4-fb2c6e55d0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:41.379464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd2c3a-1076-46e5-8cf4-fb2c6e55d0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:41.379800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd2c3a-1076-46e5-8cf4-fb2c6e55d0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:41.380079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd2c3a-1076-46e5-8cf4-fb2c6e55d0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:41.381869 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dabd2c3a-1076-46e5-8cf4-fb2c6e55d0d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:41.422959 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df6a43-f123-4be2-a35a-bbf3889b6d4a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:41.429288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:41.430159 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:41.430441 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:41.430723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] PATH_INFO: `/v3/projects/29fc69c9c1c04a64ac1578af40ffe9b6` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:41.431943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] RBAC: Authorizing `identity:delete_project(project_id=29fc69c9c1c04a64ac1578af40ffe9b6)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:41.434004 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:41.437196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:41.455642 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd2c3a-1076-46e5-8cf4-fb2c6e55d0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:41.459240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd2c3a-1076-46e5-8cf4-fb2c6e55d0d2 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:41.468320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1265/2536] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:02:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:02:41.589654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29fc69c9c1c04a64ac1578af40ffe9b6', 'request_id': 'req-22bde319-0a77-4306-a6eb-7cc495b2908b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:41.598500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29fc69c9c1c04a64ac1578af40ffe9b6', 'request_id': 'req-22bde319-0a77-4306-a6eb-7cc495b2908b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:41.604917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29fc69c9c1c04a64ac1578af40ffe9b6', 'request_id': 'req-22bde319-0a77-4306-a6eb-7cc495b2908b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:41.701239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] The token cache is being invalidate because project 29fc69c9c1c04a64ac1578af40ffe9b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:02:41.701529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:41.702302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:41.703875 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1272/2537] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:02:41 2022] DELETE /identity/v3/projects/29fc69c9c1c04a64ac1578af40ffe9b6 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:02:41.708281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd2c3a-1076-46e5-8cf4-fb2c6e55d0d2 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:41.718960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd2c3a-1076-46e5-8cf4-fb2c6e55d0d2 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:41.763594 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:41.780404 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:41.781299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:41.781578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:41.781852 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] PATH_INFO: `/v3/users/35a517ad6e5442b7a673d1caf1dd2d7a` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:41.783099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35a517ad6e5442b7a673d1caf1dd2d7a)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:41.785081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:41.999538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35a517ad6e5442b7a673d1caf1dd2d7a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:42.000199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35a517ad6e5442b7a673d1caf1dd2d7a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:42.013226 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35a517ad6e5442b7a673d1caf1dd2d7a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:42.015008 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35a517ad6e5442b7a673d1caf1dd2d7a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:42.022797 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35a517ad6e5442b7a673d1caf1dd2d7a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:42.087565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1266/2538] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:02:41 2022] DELETE /identity/v3/users/35a517ad6e5442b7a673d1caf1dd2d7a => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:02:42.346361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:42.347970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:42.384330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bde319-0a77-4306-a6eb-7cc495b2908b admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:42.438576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:42.439439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:42.439726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:42.440006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] PATH_INFO: `/v3/projects/e8f61d64d47e4978a12339c665c6a4f6` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:42.441242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8f61d64d47e4978a12339c665c6a4f6)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:42.443156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:42.608837 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8f61d64d47e4978a12339c665c6a4f6', 'request_id': 'req-92475575-378f-49fc-8633-f8f56ffaba86'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:42.615082 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8f61d64d47e4978a12339c665c6a4f6', 'request_id': 'req-92475575-378f-49fc-8633-f8f56ffaba86'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:42.621195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8f61d64d47e4978a12339c665c6a4f6', 'request_id': 'req-92475575-378f-49fc-8633-f8f56ffaba86'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:42.707889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] The token cache is being invalidate because project e8f61d64d47e4978a12339c665c6a4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:02:42.708545 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:42.709485 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:42.714611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1273/2539] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:02:42 2022] DELETE /identity/v3/projects/e8f61d64d47e4978a12339c665c6a4f6 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:02:45.162691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:45.164161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:45.180462 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff9a83-bc00-4231-b1ef-8bae62a7c449 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:45.240781 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:45.241912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:45.242193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:45.242514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] PATH_INFO: `/v3/users/83bb340d10e44bedad5fc9ddaf332552` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:45.243693 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bb340d10e44bedad5fc9ddaf332552)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:45.245638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:45.427930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bb340d10e44bedad5fc9ddaf332552'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:45.428658 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bb340d10e44bedad5fc9ddaf332552'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:45.440691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:45.441895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bb340d10e44bedad5fc9ddaf332552'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:45.442564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bb340d10e44bedad5fc9ddaf332552'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:45.449638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08f29e2b-58be-4265-a55e-b82505170216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bb340d10e44bedad5fc9ddaf332552'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:45.521429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1267/2540] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:02:45 2022] DELETE /identity/v3/users/83bb340d10e44bedad5fc9ddaf332552 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:02:45.787076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:45.788267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:45.802075 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92475575-378f-49fc-8633-f8f56ffaba86 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:45.823537 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:45.861577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:45.862453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:45.862740 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:45.863020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] PATH_INFO: `/v3/projects/fc7d718373b94bd3b0c0ff86cd0975bd` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:45.864209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7d718373b94bd3b0c0ff86cd0975bd)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:45.866156 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:45.989620 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7d718373b94bd3b0c0ff86cd0975bd', 'request_id': 'req-321d3bc7-846e-4adf-947e-7aea2896f90b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:45.996059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7d718373b94bd3b0c0ff86cd0975bd', 'request_id': 'req-321d3bc7-846e-4adf-947e-7aea2896f90b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:46.002018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7d718373b94bd3b0c0ff86cd0975bd', 'request_id': 'req-321d3bc7-846e-4adf-947e-7aea2896f90b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:46.074874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] The token cache is being invalidate because project fc7d718373b94bd3b0c0ff86cd0975bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:02:46.075165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:46.075782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-321d3bc7-846e-4adf-947e-7aea2896f90b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:46.077968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1274/2541] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:02:45 2022] DELETE /identity/v3/projects/fc7d718373b94bd3b0c0ff86cd0975bd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:02:46.082921 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb1106-5e50-4261-a047-61dabc155144 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.083237 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb1106-5e50-4261-a047-61dabc155144 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.083520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb1106-5e50-4261-a047-61dabc155144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.101638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9aeb1106-5e50-4261-a047-61dabc155144 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:02:46.102842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9aeb1106-5e50-4261-a047-61dabc155144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:46.163150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1268/2542] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:02:46.171063 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d1211-b6b3-4d2c-a75c-c2e19d0e024f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.171360 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d1211-b6b3-4d2c-a75c-c2e19d0e024f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.171645 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d1211-b6b3-4d2c-a75c-c2e19d0e024f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.190507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-912d1211-b6b3-4d2c-a75c-c2e19d0e024f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:02:46.192883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-912d1211-b6b3-4d2c-a75c-c2e19d0e024f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:46.200583 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1275/2543] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:02:46.205198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb1106-5e50-4261-a047-61dabc155144 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:46.214929 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb1106-5e50-4261-a047-61dabc155144 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:46.219615 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0b4d5-1973-4eb8-a753-6f947b5e09b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:46.221689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0b4d5-1973-4eb8-a753-6f947b5e09b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.221689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0b4d5-1973-4eb8-a753-6f947b5e09b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.221689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0b4d5-1973-4eb8-a753-6f947b5e09b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.221689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0b4d5-1973-4eb8-a753-6f947b5e09b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:02:46.222054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0b4d5-1973-4eb8-a753-6f947b5e09b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:46.223422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0b4d5-1973-4eb8-a753-6f947b5e09b1 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:46.234881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1269/2544] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:02:46.238978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d1211-b6b3-4d2c-a75c-c2e19d0e024f None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:46.249119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d1211-b6b3-4d2c-a75c-c2e19d0e024f None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:46.253871 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c601c-259f-4204-a9e8-26ebcbd2c2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:46.254532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c601c-259f-4204-a9e8-26ebcbd2c2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.254825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c601c-259f-4204-a9e8-26ebcbd2c2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.255108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c601c-259f-4204-a9e8-26ebcbd2c2f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.255811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c601c-259f-4204-a9e8-26ebcbd2c2f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:46.257751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c601c-259f-4204-a9e8-26ebcbd2c2f3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:46.339725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1276/2545] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:02:46 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:02:46.343844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0b4d5-1973-4eb8-a753-6f947b5e09b1 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:46.354632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0b4d5-1973-4eb8-a753-6f947b5e09b1 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:46.410899 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e904fa-e8da-46c1-95b3-2c4d8c6c726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:46.411567 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e904fa-e8da-46c1-95b3-2c4d8c6c726d admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.411857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e904fa-e8da-46c1-95b3-2c4d8c6c726d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.412138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e904fa-e8da-46c1-95b3-2c4d8c6c726d admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.413166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e904fa-e8da-46c1-95b3-2c4d8c6c726d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:46.416250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e904fa-e8da-46c1-95b3-2c4d8c6c726d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:46.494868 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1270/2546] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:02:46 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:02:46.499790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c601c-259f-4204-a9e8-26ebcbd2c2f3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:46.510171 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c601c-259f-4204-a9e8-26ebcbd2c2f3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:46.514824 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df99791-6adb-417e-9c64-f6bce3954207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:46.515484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99791-6adb-417e-9c64-f6bce3954207 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.515772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99791-6adb-417e-9c64-f6bce3954207 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.516053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99791-6adb-417e-9c64-f6bce3954207 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.516825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df99791-6adb-417e-9c64-f6bce3954207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:46.518427 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df99791-6adb-417e-9c64-f6bce3954207 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:46.528230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1277/2547] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:02:46.532088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e904fa-e8da-46c1-95b3-2c4d8c6c726d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:46.541935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e904fa-e8da-46c1-95b3-2c4d8c6c726d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:46.547648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682790b-4994-4600-abe7-628d78464a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:46.548345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682790b-4994-4600-abe7-628d78464a1e admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.548669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682790b-4994-4600-abe7-628d78464a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.548952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682790b-4994-4600-abe7-628d78464a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.549678 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682790b-4994-4600-abe7-628d78464a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:46.551305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682790b-4994-4600-abe7-628d78464a1e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:46.561838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1271/2548] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:02:46.565718 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df99791-6adb-417e-9c64-f6bce3954207 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:46.575926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df99791-6adb-417e-9c64-f6bce3954207 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:46.580615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8448a4d-c921-4418-b3c2-76ab04696d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:46.581491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8448a4d-c921-4418-b3c2-76ab04696d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.581778 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8448a4d-c921-4418-b3c2-76ab04696d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.582058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8448a4d-c921-4418-b3c2-76ab04696d4d admin admin] PATH_INFO: `/v3/projects/aa6ffdf7690f4e01b3c942dc9bde5ae7/users/d8d44b4a62484af29eec02ccee156363/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.596298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8448a4d-c921-4418-b3c2-76ab04696d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa6ffdf7690f4e01b3c942dc9bde5ae7, user_id=d8d44b4a62484af29eec02ccee156363, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:46.599013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8448a4d-c921-4418-b3c2-76ab04696d4d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:46.658434 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1278/2549] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:02:46 2022] PUT /identity/v3/projects/aa6ffdf7690f4e01b3c942dc9bde5ae7/users/d8d44b4a62484af29eec02ccee156363/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:02:46.669109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5b8af-7007-4e5f-9983-f5f127bca5a0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.669402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5b8af-7007-4e5f-9983-f5f127bca5a0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.669684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5b8af-7007-4e5f-9983-f5f127bca5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.696641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-84e5b8af-7007-4e5f-9983-f5f127bca5a0 None None] MFA Rules not processed for user `d8d44b4a62484af29eec02ccee156363`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:02:46.697850 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-84e5b8af-7007-4e5f-9983-f5f127bca5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:46.730480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:46.761926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1272/2550] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:02:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:02:46.766070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8448a4d-c921-4418-b3c2-76ab04696d4d admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:46.809217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:46.827806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8448a4d-c921-4418-b3c2-76ab04696d4d admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:46.832408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72c4a7-6a89-4ddd-a826-c848b7e646d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:46.833070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72c4a7-6a89-4ddd-a826-c848b7e646d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.833356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72c4a7-6a89-4ddd-a826-c848b7e646d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.833639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72c4a7-6a89-4ddd-a826-c848b7e646d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.834323 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72c4a7-6a89-4ddd-a826-c848b7e646d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:46.836219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72c4a7-6a89-4ddd-a826-c848b7e646d4 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:46.905756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1279/2551] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:02:46 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:02:46.910874 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5b8af-7007-4e5f-9983-f5f127bca5a0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:46.922243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5b8af-7007-4e5f-9983-f5f127bca5a0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:46.977863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696f404-0430-45a2-a566-95bf06f2c923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:46.978523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696f404-0430-45a2-a566-95bf06f2c923 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:46.978810 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696f404-0430-45a2-a566-95bf06f2c923 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:46.979088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696f404-0430-45a2-a566-95bf06f2c923 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:46.979787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696f404-0430-45a2-a566-95bf06f2c923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:46.981736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696f404-0430-45a2-a566-95bf06f2c923 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:47.071575 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1273/2552] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:02:46 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:02:47.075766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72c4a7-6a89-4ddd-a826-c848b7e646d4 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:47.087003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72c4a7-6a89-4ddd-a826-c848b7e646d4 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:47.091662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e7a0f-1973-42b1-9a91-7398654e63dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:47.093032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e7a0f-1973-42b1-9a91-7398654e63dd admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:47.093299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e7a0f-1973-42b1-9a91-7398654e63dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:47.093575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e7a0f-1973-42b1-9a91-7398654e63dd admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:47.094293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e7a0f-1973-42b1-9a91-7398654e63dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:47.095874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e7a0f-1973-42b1-9a91-7398654e63dd admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:47.106345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1280/2553] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:02:47.110379 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696f404-0430-45a2-a566-95bf06f2c923 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:47.120317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696f404-0430-45a2-a566-95bf06f2c923 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:47.124842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3479fe-36c8-4d10-8e2e-db253317d0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:47.125716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3479fe-36c8-4d10-8e2e-db253317d0dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:47.125996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3479fe-36c8-4d10-8e2e-db253317d0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:47.126267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3479fe-36c8-4d10-8e2e-db253317d0dc admin admin] PATH_INFO: `/v3/projects/abbdc9ce832c4467a851fefebf58acb8/users/da1882718bed4fb3835f532fa4f6cafb/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:47.141241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3479fe-36c8-4d10-8e2e-db253317d0dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=abbdc9ce832c4467a851fefebf58acb8, user_id=da1882718bed4fb3835f532fa4f6cafb, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:47.143872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3479fe-36c8-4d10-8e2e-db253317d0dc admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:47.238349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1274/2554] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:02:47 2022] PUT /identity/v3/projects/abbdc9ce832c4467a851fefebf58acb8/users/da1882718bed4fb3835f532fa4f6cafb/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:02:47.242578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e7a0f-1973-42b1-9a91-7398654e63dd admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:47.252903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e7a0f-1973-42b1-9a91-7398654e63dd admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:47.311897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde6ebe-9266-42a0-950a-fb1c5b840ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:47.312611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6ebe-9266-42a0-950a-fb1c5b840ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:47.312897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6ebe-9266-42a0-950a-fb1c5b840ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:47.313174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6ebe-9266-42a0-950a-fb1c5b840ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:47.313947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde6ebe-9266-42a0-950a-fb1c5b840ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:47.315538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde6ebe-9266-42a0-950a-fb1c5b840ec6 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:47.325322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1281/2555] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:02:47.329824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3479fe-36c8-4d10-8e2e-db253317d0dc admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:47.340140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3479fe-36c8-4d10-8e2e-db253317d0dc admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:47.345427 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45112dd8-1669-4e4b-9802-06e169a061c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:47.346320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45112dd8-1669-4e4b-9802-06e169a061c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:47.346584 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45112dd8-1669-4e4b-9802-06e169a061c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:47.346855 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45112dd8-1669-4e4b-9802-06e169a061c2 admin admin] PATH_INFO: `/v3/projects/abbdc9ce832c4467a851fefebf58acb8/users/da1882718bed4fb3835f532fa4f6cafb/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:47.348839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45112dd8-1669-4e4b-9802-06e169a061c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abbdc9ce832c4467a851fefebf58acb8, user_id=da1882718bed4fb3835f532fa4f6cafb, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:47.351450 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45112dd8-1669-4e4b-9802-06e169a061c2 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:47.362432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1275/2556] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:02:47 2022] PUT /identity/v3/projects/abbdc9ce832c4467a851fefebf58acb8/users/da1882718bed4fb3835f532fa4f6cafb/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:02:47.372776 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c71a-cd6e-453e-a6d9-c3a276f36be3 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:47.373073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c71a-cd6e-453e-a6d9-c3a276f36be3 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:47.373394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c71a-cd6e-453e-a6d9-c3a276f36be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:47.402218 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-eb63c71a-cd6e-453e-a6d9-c3a276f36be3 None None] MFA Rules not processed for user `da1882718bed4fb3835f532fa4f6cafb`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:02:47.421684 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-eb63c71a-cd6e-453e-a6d9-c3a276f36be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:47.471611 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1282/2557] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:02:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:02:47.482997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45112dd8-1669-4e4b-9802-06e169a061c2 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:47.484727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-45112dd8-1669-4e4b-9802-06e169a061c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:47.542142 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45112dd8-1669-4e4b-9802-06e169a061c2 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:47.545716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88880d19-d28d-454b-a2cf-f22f02ad7a6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:47.546355 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88880d19-d28d-454b-a2cf-f22f02ad7a6f service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:47.546636 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88880d19-d28d-454b-a2cf-f22f02ad7a6f service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:47.546910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88880d19-d28d-454b-a2cf-f22f02ad7a6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:47.556147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88880d19-d28d-454b-a2cf-f22f02ad7a6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:47.557603 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88880d19-d28d-454b-a2cf-f22f02ad7a6f service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:47.567356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1276/2558] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:02:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:02:47.790086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c71a-cd6e-453e-a6d9-c3a276f36be3 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:47.799707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c71a-cd6e-453e-a6d9-c3a276f36be3 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:47.803311 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037773e8-e7e3-476b-b87f-b5f35f921c1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:47.803948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037773e8-e7e3-476b-b87f-b5f35f921c1b service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:47.804234 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037773e8-e7e3-476b-b87f-b5f35f921c1b service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:47.804558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037773e8-e7e3-476b-b87f-b5f35f921c1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:47.837538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:47.862202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037773e8-e7e3-476b-b87f-b5f35f921c1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:47.863928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037773e8-e7e3-476b-b87f-b5f35f921c1b service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:47.874225 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1283/2559] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:02:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:02:57.563310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:57.894690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=12 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:58.336984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88880d19-d28d-454b-a2cf-f22f02ad7a6f service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:58.400711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88880d19-d28d-454b-a2cf-f22f02ad7a6f service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:58.405286 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:58.406152 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:58.406440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:58.406713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] PATH_INFO: `/v3/users/d8d44b4a62484af29eec02ccee156363` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:58.408608 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d44b4a62484af29eec02ccee156363)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:58.410530 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:58.536705 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d44b4a62484af29eec02ccee156363'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:58.537272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d44b4a62484af29eec02ccee156363'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:58.548803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d44b4a62484af29eec02ccee156363'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:58.549449 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d44b4a62484af29eec02ccee156363'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:58.556332 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d44b4a62484af29eec02ccee156363'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:58.637212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1277/2560] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:02:58 2022] DELETE /identity/v3/users/d8d44b4a62484af29eec02ccee156363 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:02:58.641697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c797af-c66b-477f-9240-75447dc2f93f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:58.641989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c797af-c66b-477f-9240-75447dc2f93f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:58.642942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c797af-c66b-477f-9240-75447dc2f93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:58.660393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b8c797af-c66b-477f-9240-75447dc2f93f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:02:58.661630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b8c797af-c66b-477f-9240-75447dc2f93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:58.721797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1284/2561] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:02:58.731475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:58.733002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:58.789617 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe0903-575b-4962-883c-b95c99b7ab01 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:58.793410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335b427-4d0d-4129-88a1-8dd15c39457e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:58.794062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335b427-4d0d-4129-88a1-8dd15c39457e service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:58.794373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335b427-4d0d-4129-88a1-8dd15c39457e service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:58.794650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335b427-4d0d-4129-88a1-8dd15c39457e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:58.803642 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0335b427-4d0d-4129-88a1-8dd15c39457e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:58.805081 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0335b427-4d0d-4129-88a1-8dd15c39457e service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:58.815035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1278/2562] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:02:58.964898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b911d2f-3722-463a-9206-cf0c309052a2 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:58.965356 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b911d2f-3722-463a-9206-cf0c309052a2 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:58.965468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b911d2f-3722-463a-9206-cf0c309052a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:58.984166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-6b911d2f-3722-463a-9206-cf0c309052a2 None None] MFA Rules not processed for user `7e03ea067e27456590c0865766830d73`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:02:58.986513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6b911d2f-3722-463a-9206-cf0c309052a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:59.050232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1285/2563] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:02:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:02:59.285510 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335b427-4d0d-4129-88a1-8dd15c39457e service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:59.288648 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-0335b427-4d0d-4129-88a1-8dd15c39457e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:02:59.306637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335b427-4d0d-4129-88a1-8dd15c39457e service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:59.306637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:59.307364 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:59.307646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:59.307919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] PATH_INFO: `/v3/projects/aa6ffdf7690f4e01b3c942dc9bde5ae7` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:59.309200 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa6ffdf7690f4e01b3c942dc9bde5ae7)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:59.311036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:59.330602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:59.430363 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa6ffdf7690f4e01b3c942dc9bde5ae7', 'request_id': 'req-3ba85241-2b94-4bdd-a8a0-7628169b45a4'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:59.437146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa6ffdf7690f4e01b3c942dc9bde5ae7', 'request_id': 'req-3ba85241-2b94-4bdd-a8a0-7628169b45a4'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:59.442941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa6ffdf7690f4e01b3c942dc9bde5ae7', 'request_id': 'req-3ba85241-2b94-4bdd-a8a0-7628169b45a4'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:59.537925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] The token cache is being invalidate because project aa6ffdf7690f4e01b3c942dc9bde5ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:02:59.537925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:59.538485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:59.540052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1279/2564] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:02:59 2022] DELETE /identity/v3/projects/aa6ffdf7690f4e01b3c942dc9bde5ae7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:02:59.544249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b911d2f-3722-463a-9206-cf0c309052a2 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:02:59.556514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b911d2f-3722-463a-9206-cf0c309052a2 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:02:59.615456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:02:59.616643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:02:59.616923 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:02:59.617197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] PATH_INFO: `/v3/users/da1882718bed4fb3835f532fa4f6cafb` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:02:59.618433 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1882718bed4fb3835f532fa4f6cafb)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:02:59.620344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:02:59.838265 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1882718bed4fb3835f532fa4f6cafb'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:59.838980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1882718bed4fb3835f532fa4f6cafb'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:59.839669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1882718bed4fb3835f532fa4f6cafb'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:59.841078 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:02:59.848017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1882718bed4fb3835f532fa4f6cafb'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:59.862078 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-80a89217-8270-44a3-854d-9d5c92ca5ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1882718bed4fb3835f532fa4f6cafb'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:02:59.945421 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1286/2565] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:02:59 2022] DELETE /identity/v3/users/da1882718bed4fb3835f532fa4f6cafb => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:03:00.333380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:00.334780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:00.348054 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba85241-2b94-4bdd-a8a0-7628169b45a4 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:00.416129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:00.417087 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:00.417369 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:00.417646 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] PATH_INFO: `/v3/projects/abbdc9ce832c4467a851fefebf58acb8` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:00.418820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abbdc9ce832c4467a851fefebf58acb8)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:00.420791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:00.551586 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abbdc9ce832c4467a851fefebf58acb8', 'request_id': 'req-55b0ba49-f24b-424f-b484-589279260361'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:00.557924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abbdc9ce832c4467a851fefebf58acb8', 'request_id': 'req-55b0ba49-f24b-424f-b484-589279260361'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:00.563615 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abbdc9ce832c4467a851fefebf58acb8', 'request_id': 'req-55b0ba49-f24b-424f-b484-589279260361'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:00.630755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] The token cache is being invalidate because project abbdc9ce832c4467a851fefebf58acb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:03:00.630984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:00.631595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-55b0ba49-f24b-424f-b484-589279260361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:00.633222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1280/2566] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:03:00 2022] DELETE /identity/v3/projects/abbdc9ce832c4467a851fefebf58acb8 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:00.638037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c0df0-41b4-44cd-82da-49bf4f20aede None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:00.638518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c0df0-41b4-44cd-82da-49bf4f20aede None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:00.638632 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c0df0-41b4-44cd-82da-49bf4f20aede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:00.656462 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-7d4c0df0-41b4-44cd-82da-49bf4f20aede None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:00.657656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7d4c0df0-41b4-44cd-82da-49bf4f20aede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:00.717332 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1287/2567] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:00.725230 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75f6dc-5c14-4a87-84b7-5e3266d09988 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:00.725527 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75f6dc-5c14-4a87-84b7-5e3266d09988 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:00.725809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75f6dc-5c14-4a87-84b7-5e3266d09988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:00.744691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-4c75f6dc-5c14-4a87-84b7-5e3266d09988 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:00.745892 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-4c75f6dc-5c14-4a87-84b7-5e3266d09988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:00.754263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1281/2568] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:00.758339 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c0df0-41b4-44cd-82da-49bf4f20aede None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:00.768212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c0df0-41b4-44cd-82da-49bf4f20aede None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:00.772823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb52516-5de2-403c-98a4-1d8797b9be17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:00.773474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb52516-5de2-403c-98a4-1d8797b9be17 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:00.773758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb52516-5de2-403c-98a4-1d8797b9be17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:00.774034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb52516-5de2-403c-98a4-1d8797b9be17 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:00.774754 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb52516-5de2-403c-98a4-1d8797b9be17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:03:00.775062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb52516-5de2-403c-98a4-1d8797b9be17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:00.776739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb52516-5de2-403c-98a4-1d8797b9be17 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:00.788324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1288/2569] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:03:00.792492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75f6dc-5c14-4a87-84b7-5e3266d09988 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:00.803174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75f6dc-5c14-4a87-84b7-5e3266d09988 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:00.807960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10f019-e0e5-4a4c-a1cc-dfcf62bac6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:00.808666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10f019-e0e5-4a4c-a1cc-dfcf62bac6ac admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:00.808987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10f019-e0e5-4a4c-a1cc-dfcf62bac6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:00.809351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10f019-e0e5-4a4c-a1cc-dfcf62bac6ac admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:00.810058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10f019-e0e5-4a4c-a1cc-dfcf62bac6ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:00.812092 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10f019-e0e5-4a4c-a1cc-dfcf62bac6ac admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:00.912139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1282/2570] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:00 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:03:00.916282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb52516-5de2-403c-98a4-1d8797b9be17 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:00.927377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb52516-5de2-403c-98a4-1d8797b9be17 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:00.986637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4f3e3-162f-40a5-81be-8271c46c0274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:00.990070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4f3e3-162f-40a5-81be-8271c46c0274 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:00.990070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4f3e3-162f-40a5-81be-8271c46c0274 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:00.990070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4f3e3-162f-40a5-81be-8271c46c0274 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:00.990070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4f3e3-162f-40a5-81be-8271c46c0274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:00.992581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4f3e3-162f-40a5-81be-8271c46c0274 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:01.077136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1289/2571] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:00 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:01.086252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10f019-e0e5-4a4c-a1cc-dfcf62bac6ac admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:01.097191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10f019-e0e5-4a4c-a1cc-dfcf62bac6ac admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:01.101853 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d997e-6858-406a-97cd-ca0766f0aa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:01.102516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d997e-6858-406a-97cd-ca0766f0aa30 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:01.102792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d997e-6858-406a-97cd-ca0766f0aa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:01.103065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d997e-6858-406a-97cd-ca0766f0aa30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:01.103789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d997e-6858-406a-97cd-ca0766f0aa30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:01.105410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d997e-6858-406a-97cd-ca0766f0aa30 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:01.106725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:01.117242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1283/2572] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:01.130647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4f3e3-162f-40a5-81be-8271c46c0274 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:01.147278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4f3e3-162f-40a5-81be-8271c46c0274 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:01.152422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da47f8-d9e0-4b2c-9413-bf2b9d8b14fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:01.153100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da47f8-d9e0-4b2c-9413-bf2b9d8b14fd admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:01.153381 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da47f8-d9e0-4b2c-9413-bf2b9d8b14fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:01.153661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da47f8-d9e0-4b2c-9413-bf2b9d8b14fd admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:01.154385 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da47f8-d9e0-4b2c-9413-bf2b9d8b14fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:01.156021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da47f8-d9e0-4b2c-9413-bf2b9d8b14fd admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:01.169430 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1290/2573] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:01.172526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d997e-6858-406a-97cd-ca0766f0aa30 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:01.184314 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d997e-6858-406a-97cd-ca0766f0aa30 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:01.189568 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9112e3-f99c-4683-add2-414a64515801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:01.190641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9112e3-f99c-4683-add2-414a64515801 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:01.191137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9112e3-f99c-4683-add2-414a64515801 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:01.191542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9112e3-f99c-4683-add2-414a64515801 admin admin] PATH_INFO: `/v3/projects/3f46101a9a694ceea550b35ee7e703d1/users/56495f29dfc14c0c98edc8f95aa31e43/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:01.208703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9112e3-f99c-4683-add2-414a64515801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f46101a9a694ceea550b35ee7e703d1, user_id=56495f29dfc14c0c98edc8f95aa31e43, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:01.211483 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9112e3-f99c-4683-add2-414a64515801 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:01.330094 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1284/2574] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:01 2022] PUT /identity/v3/projects/3f46101a9a694ceea550b35ee7e703d1/users/56495f29dfc14c0c98edc8f95aa31e43/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:01.341165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0de4-494a-485a-8a07-f4582f88d2a2 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:01.341459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0de4-494a-485a-8a07-f4582f88d2a2 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:01.341743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0de4-494a-485a-8a07-f4582f88d2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:01.368996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-88be0de4-494a-485a-8a07-f4582f88d2a2 None None] MFA Rules not processed for user `56495f29dfc14c0c98edc8f95aa31e43`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:01.370213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-88be0de4-494a-485a-8a07-f4582f88d2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:01.388193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:01.436614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1291/2575] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:01.441022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9112e3-f99c-4683-add2-414a64515801 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:01.503870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9112e3-f99c-4683-add2-414a64515801 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:01.508616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e52a2-cb4e-4c52-ae08-1d33c3f8008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:01.509276 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e52a2-cb4e-4c52-ae08-1d33c3f8008c admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:01.509566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e52a2-cb4e-4c52-ae08-1d33c3f8008c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:01.509846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e52a2-cb4e-4c52-ae08-1d33c3f8008c admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:01.510551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e52a2-cb4e-4c52-ae08-1d33c3f8008c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:01.512528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e52a2-cb4e-4c52-ae08-1d33c3f8008c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:01.580988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1285/2576] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:01 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:01.585066 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be0de4-494a-485a-8a07-f4582f88d2a2 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:01.596325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be0de4-494a-485a-8a07-f4582f88d2a2 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:01.654350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4bfbd-e922-4525-ae83-e0ecfae5d012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:01.655063 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4bfbd-e922-4525-ae83-e0ecfae5d012 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:01.655299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4bfbd-e922-4525-ae83-e0ecfae5d012 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:01.655566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4bfbd-e922-4525-ae83-e0ecfae5d012 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:01.656252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4bfbd-e922-4525-ae83-e0ecfae5d012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:01.658255 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4bfbd-e922-4525-ae83-e0ecfae5d012 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:01.792813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1292/2577] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:01 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:01.795294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e52a2-cb4e-4c52-ae08-1d33c3f8008c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:01.805455 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e52a2-cb4e-4c52-ae08-1d33c3f8008c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:01.809926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d96c7-af0d-43b7-af83-93b81f9755bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:01.810587 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d96c7-af0d-43b7-af83-93b81f9755bc admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:01.810890 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d96c7-af0d-43b7-af83-93b81f9755bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:01.811243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d96c7-af0d-43b7-af83-93b81f9755bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:01.811971 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d96c7-af0d-43b7-af83-93b81f9755bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:01.813606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d96c7-af0d-43b7-af83-93b81f9755bc admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:01.824578 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1286/2578] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:01.828634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4bfbd-e922-4525-ae83-e0ecfae5d012 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:01.839173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4bfbd-e922-4525-ae83-e0ecfae5d012 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:01.843798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f7c34-548c-4ad0-8a4f-a4c72337997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:01.844705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f7c34-548c-4ad0-8a4f-a4c72337997b admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:01.844985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f7c34-548c-4ad0-8a4f-a4c72337997b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:01.845256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f7c34-548c-4ad0-8a4f-a4c72337997b admin admin] PATH_INFO: `/v3/projects/06717eb4670e4de3a3a8774b96f596be/users/942e2c66aa5c49c1a0cd8bb77c967b01/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:01.859867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f7c34-548c-4ad0-8a4f-a4c72337997b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06717eb4670e4de3a3a8774b96f596be, user_id=942e2c66aa5c49c1a0cd8bb77c967b01, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:01.862604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f7c34-548c-4ad0-8a4f-a4c72337997b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:01.974312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1293/2579] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:01 2022] PUT /identity/v3/projects/06717eb4670e4de3a3a8774b96f596be/users/942e2c66aa5c49c1a0cd8bb77c967b01/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:01.978220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d96c7-af0d-43b7-af83-93b81f9755bc admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:01.988627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d96c7-af0d-43b7-af83-93b81f9755bc admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:02.047378 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce25876-acf8-4182-95e1-b50486ce057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:02.048031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce25876-acf8-4182-95e1-b50486ce057c admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:02.048900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce25876-acf8-4182-95e1-b50486ce057c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:02.049177 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce25876-acf8-4182-95e1-b50486ce057c admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:02.050005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce25876-acf8-4182-95e1-b50486ce057c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:02.051636 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce25876-acf8-4182-95e1-b50486ce057c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:02.066219 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1287/2580] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:02.067417 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f7c34-548c-4ad0-8a4f-a4c72337997b admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:02.078119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f7c34-548c-4ad0-8a4f-a4c72337997b admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:02.082918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3e538-e495-47c6-9837-804c15e9d825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:02.083791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3e538-e495-47c6-9837-804c15e9d825 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:02.084079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3e538-e495-47c6-9837-804c15e9d825 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:02.084353 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3e538-e495-47c6-9837-804c15e9d825 admin admin] PATH_INFO: `/v3/projects/06717eb4670e4de3a3a8774b96f596be/users/942e2c66aa5c49c1a0cd8bb77c967b01/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:02.086416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3e538-e495-47c6-9837-804c15e9d825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06717eb4670e4de3a3a8774b96f596be, user_id=942e2c66aa5c49c1a0cd8bb77c967b01, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:02.089397 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3e538-e495-47c6-9837-804c15e9d825 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:02.107627 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1294/2581] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:02 2022] PUT /identity/v3/projects/06717eb4670e4de3a3a8774b96f596be/users/942e2c66aa5c49c1a0cd8bb77c967b01/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:03:02.113258 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74006d15-c6f3-4f4d-a182-5152681b1ad1 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:02.113522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74006d15-c6f3-4f4d-a182-5152681b1ad1 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:02.113787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74006d15-c6f3-4f4d-a182-5152681b1ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:02.145645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-74006d15-c6f3-4f4d-a182-5152681b1ad1 None None] MFA Rules not processed for user `942e2c66aa5c49c1a0cd8bb77c967b01`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:02.146637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-74006d15-c6f3-4f4d-a182-5152681b1ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:02.181621 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:02.225195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1288/2582] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:02.237919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3e538-e495-47c6-9837-804c15e9d825 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:02.239943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-8ef3e538-e495-47c6-9837-804c15e9d825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:02.317329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3e538-e495-47c6-9837-804c15e9d825 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:02.321007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdb48a-9e90-4124-b3d8-365499091333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:02.321662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdb48a-9e90-4124-b3d8-365499091333 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:02.321941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdb48a-9e90-4124-b3d8-365499091333 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:02.322226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdb48a-9e90-4124-b3d8-365499091333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:02.357643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:02.390194 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdb48a-9e90-4124-b3d8-365499091333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:02.391659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdb48a-9e90-4124-b3d8-365499091333 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:02.401000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1295/2583] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:07.634248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74006d15-c6f3-4f4d-a182-5152681b1ad1 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:07.644570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74006d15-c6f3-4f4d-a182-5152681b1ad1 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:07.648156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fb7a3-9bd2-4e93-b504-82d635cf2b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:07.648847 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb7a3-9bd2-4e93-b504-82d635cf2b3f service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:07.649134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb7a3-9bd2-4e93-b504-82d635cf2b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:07.649413 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fb7a3-9bd2-4e93-b504-82d635cf2b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:07.658330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fb7a3-9bd2-4e93-b504-82d635cf2b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:07.659747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fb7a3-9bd2-4e93-b504-82d635cf2b3f service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:07.669128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1289/2584] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:12.499529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=15 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:17.665045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=21 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:18.769024 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd736ae3-8e95-41d6-b8ff-41bcdce7659d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:18.769543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd736ae3-8e95-41d6-b8ff-41bcdce7659d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:18.769650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd736ae3-8e95-41d6-b8ff-41bcdce7659d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:18.787694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-dd736ae3-8e95-41d6-b8ff-41bcdce7659d None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:18.788921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-dd736ae3-8e95-41d6-b8ff-41bcdce7659d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:18.844644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1296/2585] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 16:03:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:18.856657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fb7a3-9bd2-4e93-b504-82d635cf2b3f service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:18.858100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a70fb7a3-9bd2-4e93-b504-82d635cf2b3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:18.927524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fb7a3-9bd2-4e93-b504-82d635cf2b3f service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:18.933043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761805ef-6b0b-45e7-a045-adda678e189e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:18.933696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761805ef-6b0b-45e7-a045-adda678e189e service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:18.933981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761805ef-6b0b-45e7-a045-adda678e189e service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:18.934255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761805ef-6b0b-45e7-a045-adda678e189e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:18.942530 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761805ef-6b0b-45e7-a045-adda678e189e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:18.944140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761805ef-6b0b-45e7-a045-adda678e189e service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:18.952878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1290/2586] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 16:03:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 16:03:28.837727 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=25 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:28.949652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=32 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:35.120532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd736ae3-8e95-41d6-b8ff-41bcdce7659d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:35.194344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd736ae3-8e95-41d6-b8ff-41bcdce7659d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:35.199851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:35.200906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.201185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.201481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] PATH_INFO: `/v3/users/56495f29dfc14c0c98edc8f95aa31e43` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.202749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=56495f29dfc14c0c98edc8f95aa31e43)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:35.204706 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:35.215229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1105a-ea7f-43ad-87c7-369aa4363740 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.220143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1105a-ea7f-43ad-87c7-369aa4363740 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.220143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1105a-ea7f-43ad-87c7-369aa4363740 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.220143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1291/2587] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 16:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:03:35.224728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e49ac-9112-49bd-9a8e-56f575eac81e None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.225253 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e49ac-9112-49bd-9a8e-56f575eac81e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.225885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e49ac-9112-49bd-9a8e-56f575eac81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.270834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-037e49ac-9112-49bd-9a8e-56f575eac81e None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:35.272072 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-037e49ac-9112-49bd-9a8e-56f575eac81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:35.341000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1292/2588] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:35.354150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e49ac-9112-49bd-9a8e-56f575eac81e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:35.355628 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-037e49ac-9112-49bd-9a8e-56f575eac81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:35.412754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e49ac-9112-49bd-9a8e-56f575eac81e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:35.416898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa5129-5ecd-481a-9526-87b7706608ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '142e6e44d6544f6a891ed3f913de74ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:35.417588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa5129-5ecd-481a-9526-87b7706608ed service glance] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.417875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa5129-5ecd-481a-9526-87b7706608ed service glance] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.418144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa5129-5ecd-481a-9526-87b7706608ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.427912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56495f29dfc14c0c98edc8f95aa31e43'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:35.428678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56495f29dfc14c0c98edc8f95aa31e43'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:35.429863 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56495f29dfc14c0c98edc8f95aa31e43'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:35.430341 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa5129-5ecd-481a-9526-87b7706608ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:35.432459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa5129-5ecd-481a-9526-87b7706608ed service glance] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:35.437466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56495f29dfc14c0c98edc8f95aa31e43'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:35.446532 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1293/2589] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 16:03:35.452054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56495f29dfc14c0c98edc8f95aa31e43'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:35.526171 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1297/2590] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:03:35 2022] DELETE /identity/v3/users/56495f29dfc14c0c98edc8f95aa31e43 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:35.530796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011b7b2-9eff-4e80-8bbf-81e198ed358c None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.531144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011b7b2-9eff-4e80-8bbf-81e198ed358c None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.531507 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011b7b2-9eff-4e80-8bbf-81e198ed358c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.549824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2011b7b2-9eff-4e80-8bbf-81e198ed358c None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:35.551047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2011b7b2-9eff-4e80-8bbf-81e198ed358c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:35.611622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1294/2591] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:35.622188 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:35.624114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:35.626088 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fb5b0-5993-4de5-8f22-a8acaa0d34c6 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.626571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fb5b0-5993-4de5-8f22-a8acaa0d34c6 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.627046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fb5b0-5993-4de5-8f22-a8acaa0d34c6 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.629452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1295/2592] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 16:03:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:03:35.636046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0cc75-de02-4467-a659-23f1f1d71d7e None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.636337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0cc75-de02-4467-a659-23f1f1d71d7e None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.636659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0cc75-de02-4467-a659-23f1f1d71d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.655659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ade0cc75-de02-4467-a659-23f1f1d71d7e None None] MFA Rules not processed for user `a39f5189d63d467dbd914f553c261c01`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:35.656910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ade0cc75-de02-4467-a659-23f1f1d71d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:35.676839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=100 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:35.697745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d065c2-bfa3-4fda-8b21-10e9a2bf36fc admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:35.701491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e69d0-ffa8-45f5-a2d6-2f4ef3cf28ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:35.702147 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e69d0-ffa8-45f5-a2d6-2f4ef3cf28ba service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.702436 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e69d0-ffa8-45f5-a2d6-2f4ef3cf28ba service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.702720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e69d0-ffa8-45f5-a2d6-2f4ef3cf28ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.712593 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e69d0-ffa8-45f5-a2d6-2f4ef3cf28ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:35.713442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1296/2593] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:03:35.717244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e69d0-ffa8-45f5-a2d6-2f4ef3cf28ba service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:35.726666 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0cc75-de02-4467-a659-23f1f1d71d7e None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:35.728224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ade0cc75-de02-4467-a659-23f1f1d71d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:35.728551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1298/2594] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:35.741029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0cc75-de02-4467-a659-23f1f1d71d7e None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:35.746520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307fb39-e02e-4085-a3c9-a2ce00b6c2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:35.750478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307fb39-e02e-4085-a3c9-a2ce00b6c2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.750478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307fb39-e02e-4085-a3c9-a2ce00b6c2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.750478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307fb39-e02e-4085-a3c9-a2ce00b6c2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.757963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307fb39-e02e-4085-a3c9-a2ce00b6c2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:35.759402 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307fb39-e02e-4085-a3c9-a2ce00b6c2f2 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:35.768204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1297/2595] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 16:03:35.855614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a5c9a-44a5-4608-b410-a12ee911531a None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.856019 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a5c9a-44a5-4608-b410-a12ee911531a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.856300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a5c9a-44a5-4608-b410-a12ee911531a None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.857826 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1299/2596] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 16:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:03:35.863573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269998-9967-4cf5-aae3-4fdd9c06a3d7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:35.864164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269998-9967-4cf5-aae3-4fdd9c06a3d7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:35.864481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269998-9967-4cf5-aae3-4fdd9c06a3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:35.884487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-5e269998-9967-4cf5-aae3-4fdd9c06a3d7 None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:35.890095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5e269998-9967-4cf5-aae3-4fdd9c06a3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:35.973350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1298/2597] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:35.984983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a5c9a-44a5-4608-b410-a12ee911531a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:35.996573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a5c9a-44a5-4608-b410-a12ee911531a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:36.000493 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4337d-f687-4c68-a24f-2798d9c676d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:36.001142 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4337d-f687-4c68-a24f-2798d9c676d8 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:36.001422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4337d-f687-4c68-a24f-2798d9c676d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:36.001695 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4337d-f687-4c68-a24f-2798d9c676d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:36.014123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4337d-f687-4c68-a24f-2798d9c676d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:36.017621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4337d-f687-4c68-a24f-2798d9c676d8 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:36.027225 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1300/2598] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 16:03:36.172796 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e269998-9967-4cf5-aae3-4fdd9c06a3d7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:36.180465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5e269998-9967-4cf5-aae3-4fdd9c06a3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:36.204287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e269998-9967-4cf5-aae3-4fdd9c06a3d7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:36.211386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:36.212525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:36.212988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:36.213421 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] PATH_INFO: `/v3/projects/3f46101a9a694ceea550b35ee7e703d1` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:36.215443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f46101a9a694ceea550b35ee7e703d1)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:36.218910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:36.382822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f46101a9a694ceea550b35ee7e703d1', 'request_id': 'req-c9733ae9-b365-4350-b214-7498587879c8'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:36.390103 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f46101a9a694ceea550b35ee7e703d1', 'request_id': 'req-c9733ae9-b365-4350-b214-7498587879c8'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:36.398752 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f46101a9a694ceea550b35ee7e703d1', 'request_id': 'req-c9733ae9-b365-4350-b214-7498587879c8'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:36.469526 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] The token cache is being invalidate because project 3f46101a9a694ceea550b35ee7e703d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:03:36.469893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:36.470616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:36.472714 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1299/2599] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:03:36 2022] DELETE /identity/v3/projects/3f46101a9a694ceea550b35ee7e703d1 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:36.476955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4337d-f687-4c68-a24f-2798d9c676d8 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:36.487901 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4337d-f687-4c68-a24f-2798d9c676d8 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:36.493355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:36.554439 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:36.555317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:36.555597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:36.555873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] PATH_INFO: `/v3/users/942e2c66aa5c49c1a0cd8bb77c967b01` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:36.557088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=942e2c66aa5c49c1a0cd8bb77c967b01)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:36.559043 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:36.773745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942e2c66aa5c49c1a0cd8bb77c967b01'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:36.774399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942e2c66aa5c49c1a0cd8bb77c967b01'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:36.775001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942e2c66aa5c49c1a0cd8bb77c967b01'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:36.781988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942e2c66aa5c49c1a0cd8bb77c967b01'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:36.793564 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-3f8a56d5-08dd-47d1-9df5-5d32386ec1b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942e2c66aa5c49c1a0cd8bb77c967b01'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:36.863204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1301/2600] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:03:36 2022] DELETE /identity/v3/users/942e2c66aa5c49c1a0cd8bb77c967b01 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:37.302848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:37.306895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:37.320618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9733ae9-b365-4350-b214-7498587879c8 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:37.353401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:37.379987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:37.380879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:37.381165 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:37.381442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] PATH_INFO: `/v3/projects/06717eb4670e4de3a3a8774b96f596be` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:37.382650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] RBAC: Authorizing `identity:delete_project(project_id=06717eb4670e4de3a3a8774b96f596be)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:37.384602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:37.526488 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06717eb4670e4de3a3a8774b96f596be', 'request_id': 'req-a41857fd-cbb5-4a96-9987-8712d628637d'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:37.534293 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06717eb4670e4de3a3a8774b96f596be', 'request_id': 'req-a41857fd-cbb5-4a96-9987-8712d628637d'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:37.543916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06717eb4670e4de3a3a8774b96f596be', 'request_id': 'req-a41857fd-cbb5-4a96-9987-8712d628637d'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:37.644832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] The token cache is being invalidate because project 06717eb4670e4de3a3a8774b96f596be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:03:37.645336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:37.646099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a41857fd-cbb5-4a96-9987-8712d628637d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:37.649128 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1300/2601] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:03:37 2022] DELETE /identity/v3/projects/06717eb4670e4de3a3a8774b96f596be => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:37.657707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d4c1d-92bb-40ed-a835-484bf808d8c4 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:37.658189 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d4c1d-92bb-40ed-a835-484bf808d8c4 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:37.658286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d4c1d-92bb-40ed-a835-484bf808d8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:37.677029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-760d4c1d-92bb-40ed-a835-484bf808d8c4 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:37.678238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-760d4c1d-92bb-40ed-a835-484bf808d8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:37.737479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1302/2602] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:37.747199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70cb26-9ed0-477e-b843-8736af9069e0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:37.747496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70cb26-9ed0-477e-b843-8736af9069e0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:37.747771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70cb26-9ed0-477e-b843-8736af9069e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:37.766650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9c70cb26-9ed0-477e-b843-8736af9069e0 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:37.767857 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9c70cb26-9ed0-477e-b843-8736af9069e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:37.776430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1301/2603] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:37.780765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d4c1d-92bb-40ed-a835-484bf808d8c4 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:37.791960 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d4c1d-92bb-40ed-a835-484bf808d8c4 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:37.796594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4de959-cf23-4386-b9b1-7e2cdeda86a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:37.797262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4de959-cf23-4386-b9b1-7e2cdeda86a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:37.797578 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4de959-cf23-4386-b9b1-7e2cdeda86a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:37.797856 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4de959-cf23-4386-b9b1-7e2cdeda86a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:37.798584 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4de959-cf23-4386-b9b1-7e2cdeda86a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:03:37.798892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4de959-cf23-4386-b9b1-7e2cdeda86a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:37.800511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4de959-cf23-4386-b9b1-7e2cdeda86a8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:37.812414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1303/2604] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:03:37.817045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70cb26-9ed0-477e-b843-8736af9069e0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:37.826839 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70cb26-9ed0-477e-b843-8736af9069e0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:37.831656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2223e8-1bbb-4bfd-a205-c9b0f4009108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:37.832318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2223e8-1bbb-4bfd-a205-c9b0f4009108 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:37.832636 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2223e8-1bbb-4bfd-a205-c9b0f4009108 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:37.832915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2223e8-1bbb-4bfd-a205-c9b0f4009108 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:37.833618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2223e8-1bbb-4bfd-a205-c9b0f4009108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:37.836173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2223e8-1bbb-4bfd-a205-c9b0f4009108 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:37.927430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1302/2605] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:37 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:37.932199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4de959-cf23-4386-b9b1-7e2cdeda86a8 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:37.942698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4de959-cf23-4386-b9b1-7e2cdeda86a8 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:37.955183 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:38.002638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0a1cb-b7d0-497d-93c0-07c6250d9c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:38.003322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0a1cb-b7d0-497d-93c0-07c6250d9c47 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:38.003609 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0a1cb-b7d0-497d-93c0-07c6250d9c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:38.003890 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0a1cb-b7d0-497d-93c0-07c6250d9c47 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:38.004612 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0a1cb-b7d0-497d-93c0-07c6250d9c47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:38.006534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0a1cb-b7d0-497d-93c0-07c6250d9c47 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:38.079929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1304/2606] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:37 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:38.084106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2223e8-1bbb-4bfd-a205-c9b0f4009108 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:38.096771 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2223e8-1bbb-4bfd-a205-c9b0f4009108 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:38.103098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effee1ca-78bb-4551-9590-ff5f55bcf1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:38.103778 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effee1ca-78bb-4551-9590-ff5f55bcf1be admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:38.104061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effee1ca-78bb-4551-9590-ff5f55bcf1be admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:38.104336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effee1ca-78bb-4551-9590-ff5f55bcf1be admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:38.105098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effee1ca-78bb-4551-9590-ff5f55bcf1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:38.106830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effee1ca-78bb-4551-9590-ff5f55bcf1be admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:38.120589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1303/2607] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 16:03:38.125110 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0a1cb-b7d0-497d-93c0-07c6250d9c47 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:38.137790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0a1cb-b7d0-497d-93c0-07c6250d9c47 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:38.144251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc32d4b-8f9f-4d2e-9279-c17fd2df0b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:38.145188 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc32d4b-8f9f-4d2e-9279-c17fd2df0b40 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:38.145603 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc32d4b-8f9f-4d2e-9279-c17fd2df0b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:38.146022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc32d4b-8f9f-4d2e-9279-c17fd2df0b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:38.147091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc32d4b-8f9f-4d2e-9279-c17fd2df0b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:38.150121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc32d4b-8f9f-4d2e-9279-c17fd2df0b40 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:38.161309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1305/2608] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:38.166275 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effee1ca-78bb-4551-9590-ff5f55bcf1be admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:38.177477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effee1ca-78bb-4551-9590-ff5f55bcf1be admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:38.182770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7edf1-7e73-4b30-9251-56f7cab190ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:38.183659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7edf1-7e73-4b30-9251-56f7cab190ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:38.183946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7edf1-7e73-4b30-9251-56f7cab190ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:38.184225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7edf1-7e73-4b30-9251-56f7cab190ce admin admin] PATH_INFO: `/v3/projects/460743be77d24fe59ceb49761415d411/users/5726f56f204d475d86981c7fa3351e06/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:38.200328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7edf1-7e73-4b30-9251-56f7cab190ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=460743be77d24fe59ceb49761415d411, user_id=5726f56f204d475d86981c7fa3351e06, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:38.203076 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7edf1-7e73-4b30-9251-56f7cab190ce admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:38.325694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1304/2609] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:38 2022] PUT /identity/v3/projects/460743be77d24fe59ceb49761415d411/users/5726f56f204d475d86981c7fa3351e06/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:38.336898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe5842-0222-4fa7-a7a6-30a5f7befa6d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:38.337373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe5842-0222-4fa7-a7a6-30a5f7befa6d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:38.337759 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe5842-0222-4fa7-a7a6-30a5f7befa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:38.367729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ebfe5842-0222-4fa7-a7a6-30a5f7befa6d None None] MFA Rules not processed for user `5726f56f204d475d86981c7fa3351e06`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:38.369033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ebfe5842-0222-4fa7-a7a6-30a5f7befa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:38.435061 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1306/2610] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:38.439786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7edf1-7e73-4b30-9251-56f7cab190ce admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:38.505285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7edf1-7e73-4b30-9251-56f7cab190ce admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:38.510651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd63c61-1bf5-457a-b429-a90b81d3d7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:38.511330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63c61-1bf5-457a-b429-a90b81d3d7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:38.511623 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63c61-1bf5-457a-b429-a90b81d3d7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:38.511909 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63c61-1bf5-457a-b429-a90b81d3d7c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:38.512664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd63c61-1bf5-457a-b429-a90b81d3d7c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:38.514573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd63c61-1bf5-457a-b429-a90b81d3d7c5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:38.518488 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:38.614497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1305/2611] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:38 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:38.618756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe5842-0222-4fa7-a7a6-30a5f7befa6d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:38.629974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe5842-0222-4fa7-a7a6-30a5f7befa6d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:38.693341 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b45df-0bae-4249-ab6c-2f30eec05bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:38.694052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b45df-0bae-4249-ab6c-2f30eec05bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:38.694308 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b45df-0bae-4249-ab6c-2f30eec05bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:38.694570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b45df-0bae-4249-ab6c-2f30eec05bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:38.695456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b45df-0bae-4249-ab6c-2f30eec05bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:38.697542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b45df-0bae-4249-ab6c-2f30eec05bf0 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:38.776653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1307/2612] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:38 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:38.781500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd63c61-1bf5-457a-b429-a90b81d3d7c5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:38.796734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd63c61-1bf5-457a-b429-a90b81d3d7c5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:38.801989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a1174-4cdc-4e62-bfa9-01fbbf3617c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:38.802951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a1174-4cdc-4e62-bfa9-01fbbf3617c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:38.803233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a1174-4cdc-4e62-bfa9-01fbbf3617c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:38.803504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a1174-4cdc-4e62-bfa9-01fbbf3617c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:38.805017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a1174-4cdc-4e62-bfa9-01fbbf3617c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:38.806653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a1174-4cdc-4e62-bfa9-01fbbf3617c0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:38.820281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1306/2613] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:38.824709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b45df-0bae-4249-ab6c-2f30eec05bf0 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:38.834785 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b45df-0bae-4249-ab6c-2f30eec05bf0 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:38.839560 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613243df-6950-47b0-a660-29cfce40ad9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:38.840524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613243df-6950-47b0-a660-29cfce40ad9e admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:38.840760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613243df-6950-47b0-a660-29cfce40ad9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:38.841039 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613243df-6950-47b0-a660-29cfce40ad9e admin admin] PATH_INFO: `/v3/projects/80430ec9cfaf47e5b476910e74f85b78/users/5ca852e383ac46aab8d4968dd73adb27/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:38.853784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:38.857996 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613243df-6950-47b0-a660-29cfce40ad9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80430ec9cfaf47e5b476910e74f85b78, user_id=5ca852e383ac46aab8d4968dd73adb27, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:38.860762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613243df-6950-47b0-a660-29cfce40ad9e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:38.926463 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1308/2614] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:38 2022] PUT /identity/v3/projects/80430ec9cfaf47e5b476910e74f85b78/users/5ca852e383ac46aab8d4968dd73adb27/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:38.930991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a1174-4cdc-4e62-bfa9-01fbbf3617c0 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:38.941584 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a1174-4cdc-4e62-bfa9-01fbbf3617c0 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:39.004172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26467e7-0d50-436e-ba78-308bea6e13d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:39.004910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26467e7-0d50-436e-ba78-308bea6e13d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:39.005198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26467e7-0d50-436e-ba78-308bea6e13d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:39.005468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26467e7-0d50-436e-ba78-308bea6e13d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:39.006223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26467e7-0d50-436e-ba78-308bea6e13d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:39.007815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26467e7-0d50-436e-ba78-308bea6e13d7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:39.018179 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1307/2615] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 16:03:39.022219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613243df-6950-47b0-a660-29cfce40ad9e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:39.033087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613243df-6950-47b0-a660-29cfce40ad9e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:39.037779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af30a2-5b46-40ee-8ef1-dfa1a1593c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:39.039219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af30a2-5b46-40ee-8ef1-dfa1a1593c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:39.039580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af30a2-5b46-40ee-8ef1-dfa1a1593c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:39.039859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af30a2-5b46-40ee-8ef1-dfa1a1593c77 admin admin] PATH_INFO: `/v3/projects/80430ec9cfaf47e5b476910e74f85b78/users/5ca852e383ac46aab8d4968dd73adb27/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:39.041892 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af30a2-5b46-40ee-8ef1-dfa1a1593c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80430ec9cfaf47e5b476910e74f85b78, user_id=5ca852e383ac46aab8d4968dd73adb27, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:39.044574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af30a2-5b46-40ee-8ef1-dfa1a1593c77 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:39.055143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1309/2616] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:39 2022] PUT /identity/v3/projects/80430ec9cfaf47e5b476910e74f85b78/users/5ca852e383ac46aab8d4968dd73adb27/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:39.066300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0aebf-218a-4a38-b8ad-b4f9a8ef4e33 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:39.066669 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0aebf-218a-4a38-b8ad-b4f9a8ef4e33 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:39.066951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0aebf-218a-4a38-b8ad-b4f9a8ef4e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:39.096027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-71e0aebf-218a-4a38-b8ad-b4f9a8ef4e33 None None] MFA Rules not processed for user `5ca852e383ac46aab8d4968dd73adb27`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:39.097368 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-71e0aebf-218a-4a38-b8ad-b4f9a8ef4e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:39.168077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1308/2617] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:39.179396 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af30a2-5b46-40ee-8ef1-dfa1a1593c77 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:39.181118 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a3af30a2-5b46-40ee-8ef1-dfa1a1593c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:39.241660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af30a2-5b46-40ee-8ef1-dfa1a1593c77 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:39.245416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c0d20-f3ca-4d17-8213-e9104039bec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:39.246084 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c0d20-f3ca-4d17-8213-e9104039bec8 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:39.246373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c0d20-f3ca-4d17-8213-e9104039bec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:39.246659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c0d20-f3ca-4d17-8213-e9104039bec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:39.256600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c0d20-f3ca-4d17-8213-e9104039bec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:39.258106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c0d20-f3ca-4d17-8213-e9104039bec8 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:39.268943 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1310/2618] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:39.298543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0aebf-218a-4a38-b8ad-b4f9a8ef4e33 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:39.358118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:39.364793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0aebf-218a-4a38-b8ad-b4f9a8ef4e33 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:39.369410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:39.370311 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:39.370595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:39.372493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] PATH_INFO: `/v3/users/5726f56f204d475d86981c7fa3351e06` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:39.372493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5726f56f204d475d86981c7fa3351e06)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:39.374101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:39.590078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5726f56f204d475d86981c7fa3351e06'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:39.591263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5726f56f204d475d86981c7fa3351e06'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:39.611653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5726f56f204d475d86981c7fa3351e06'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:39.612827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5726f56f204d475d86981c7fa3351e06'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:39.622523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5726f56f204d475d86981c7fa3351e06'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:39.692185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1309/2619] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:03:39 2022] DELETE /identity/v3/users/5726f56f204d475d86981c7fa3351e06 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:39.697726 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebdff6-9d99-43f6-9b2e-e10e04756b26 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:39.698483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebdff6-9d99-43f6-9b2e-e10e04756b26 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:39.698719 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebdff6-9d99-43f6-9b2e-e10e04756b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:39.720045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-15ebdff6-9d99-43f6-9b2e-e10e04756b26 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:39.721328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-15ebdff6-9d99-43f6-9b2e-e10e04756b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:39.784716 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1311/2620] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:39.794478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:39.795886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:39.861452 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebe0f1-340b-407d-bede-bb618195e902 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:39.865328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045da3e6-97b4-4148-9aef-b763745b6532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:39.865982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045da3e6-97b4-4148-9aef-b763745b6532 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:39.866759 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045da3e6-97b4-4148-9aef-b763745b6532 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:39.867078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045da3e6-97b4-4148-9aef-b763745b6532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:39.876556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045da3e6-97b4-4148-9aef-b763745b6532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:39.877987 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045da3e6-97b4-4148-9aef-b763745b6532 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:39.888338 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1310/2621] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:40.002979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfe5f4-ca1d-4a4a-ad96-43e607ded5cd None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:40.002979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfe5f4-ca1d-4a4a-ad96-43e607ded5cd None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:40.002979 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfe5f4-ca1d-4a4a-ad96-43e607ded5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:40.031106 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-87bfe5f4-ca1d-4a4a-ad96-43e607ded5cd None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:40.033338 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-87bfe5f4-ca1d-4a4a-ad96-43e607ded5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:40.109532 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1312/2622] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 16:03:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:03:40.136990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045da3e6-97b4-4148-9aef-b763745b6532 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:40.139004 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-045da3e6-97b4-4148-9aef-b763745b6532 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:40.220041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045da3e6-97b4-4148-9aef-b763745b6532 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:40.226056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab167519-ecf2-4341-b9f6-72a750d9a104 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:40.226955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab167519-ecf2-4341-b9f6-72a750d9a104 service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:40.227483 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab167519-ecf2-4341-b9f6-72a750d9a104 service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:40.227976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab167519-ecf2-4341-b9f6-72a750d9a104 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:40.241208 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab167519-ecf2-4341-b9f6-72a750d9a104 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:40.245319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab167519-ecf2-4341-b9f6-72a750d9a104 service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:40.257543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1311/2623] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 16:03:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 16:03:40.485329 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfe5f4-ca1d-4a4a-ad96-43e607ded5cd None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:40.487259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-87bfe5f4-ca1d-4a4a-ad96-43e607ded5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:40.515245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfe5f4-ca1d-4a4a-ad96-43e607ded5cd None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:40.523080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:40.529538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:40.530054 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:40.531993 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] PATH_INFO: `/v3/projects/460743be77d24fe59ceb49761415d411` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:40.534382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=460743be77d24fe59ceb49761415d411)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:40.537740 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:40.577108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:40.681566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '460743be77d24fe59ceb49761415d411', 'request_id': 'req-fc4e7ffa-d007-45f9-8948-c5c08079d964'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:40.689131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '460743be77d24fe59ceb49761415d411', 'request_id': 'req-fc4e7ffa-d007-45f9-8948-c5c08079d964'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:40.700143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '460743be77d24fe59ceb49761415d411', 'request_id': 'req-fc4e7ffa-d007-45f9-8948-c5c08079d964'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:40.763972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] The token cache is being invalidate because project 460743be77d24fe59ceb49761415d411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:03:40.764751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:40.765806 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:40.768900 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1313/2624] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:03:40 2022] DELETE /identity/v3/projects/460743be77d24fe59ceb49761415d411 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:40.775465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab167519-ecf2-4341-b9f6-72a750d9a104 service swift] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:40.792910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab167519-ecf2-4341-b9f6-72a750d9a104 service swift] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:40.799300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:40.886385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:40.887285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:40.887753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:40.888253 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] PATH_INFO: `/v3/users/5ca852e383ac46aab8d4968dd73adb27` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:40.889902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca852e383ac46aab8d4968dd73adb27)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:40.895155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:41.112225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca852e383ac46aab8d4968dd73adb27'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:41.113499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca852e383ac46aab8d4968dd73adb27'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:41.149134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca852e383ac46aab8d4968dd73adb27'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:41.149902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca852e383ac46aab8d4968dd73adb27'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:41.158761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-868bb0a0-082b-40e6-962d-bc95ecd3de0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca852e383ac46aab8d4968dd73adb27'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:41.267239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1312/2625] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:03:40 2022] DELETE /identity/v3/users/5ca852e383ac46aab8d4968dd73adb27 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:41.869454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:41.871616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:41.886469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e7ffa-d007-45f9-8948-c5c08079d964 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:41.976767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:41.977704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:41.977988 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:41.978271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] PATH_INFO: `/v3/projects/80430ec9cfaf47e5b476910e74f85b78` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:41.979574 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] RBAC: Authorizing `identity:delete_project(project_id=80430ec9cfaf47e5b476910e74f85b78)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:41.981599 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:42.141713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80430ec9cfaf47e5b476910e74f85b78', 'request_id': 'req-33df657f-aeb1-4581-8a4e-09cef931a9af'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:42.150563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80430ec9cfaf47e5b476910e74f85b78', 'request_id': 'req-33df657f-aeb1-4581-8a4e-09cef931a9af'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:42.157334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80430ec9cfaf47e5b476910e74f85b78', 'request_id': 'req-33df657f-aeb1-4581-8a4e-09cef931a9af'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:42.235167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] The token cache is being invalidate because project 80430ec9cfaf47e5b476910e74f85b78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:03:42.235602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:42.236630 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-33df657f-aeb1-4581-8a4e-09cef931a9af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:42.240408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1314/2626] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:03:41 2022] DELETE /identity/v3/projects/80430ec9cfaf47e5b476910e74f85b78 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:42.246233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b98d62-07c0-495f-96b9-a3501ce24ba8 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:42.246805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b98d62-07c0-495f-96b9-a3501ce24ba8 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:42.247190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b98d62-07c0-495f-96b9-a3501ce24ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:42.268671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-51b98d62-07c0-495f-96b9-a3501ce24ba8 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:42.274919 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-51b98d62-07c0-495f-96b9-a3501ce24ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:42.349834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1313/2627] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:03:42.358550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cd1b3-beb9-428c-a1e2-a66515aacd59 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:42.359060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cd1b3-beb9-428c-a1e2-a66515aacd59 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:42.359462 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cd1b3-beb9-428c-a1e2-a66515aacd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:42.389781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-5c1cd1b3-beb9-428c-a1e2-a66515aacd59 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:42.391716 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5c1cd1b3-beb9-428c-a1e2-a66515aacd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:42.407917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1315/2628] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:42.409236 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b98d62-07c0-495f-96b9-a3501ce24ba8 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:42.419436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b98d62-07c0-495f-96b9-a3501ce24ba8 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:42.424123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c79665-f1cd-44b5-ae41-6775b4e9aa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:42.424906 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c79665-f1cd-44b5-ae41-6775b4e9aa21 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:42.425195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c79665-f1cd-44b5-ae41-6775b4e9aa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:42.425476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c79665-f1cd-44b5-ae41-6775b4e9aa21 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:42.426222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c79665-f1cd-44b5-ae41-6775b4e9aa21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:03:42.426523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c79665-f1cd-44b5-ae41-6775b4e9aa21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:42.428121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c79665-f1cd-44b5-ae41-6775b4e9aa21 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:42.442000 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1314/2629] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:03:42.447077 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cd1b3-beb9-428c-a1e2-a66515aacd59 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:42.457586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cd1b3-beb9-428c-a1e2-a66515aacd59 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:42.463737 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfeb3e-3539-47c9-bd38-49d88310cdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:42.465791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfeb3e-3539-47c9-bd38-49d88310cdc6 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:42.465791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfeb3e-3539-47c9-bd38-49d88310cdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:42.465791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfeb3e-3539-47c9-bd38-49d88310cdc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:42.465791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfeb3e-3539-47c9-bd38-49d88310cdc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:42.467749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfeb3e-3539-47c9-bd38-49d88310cdc6 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:42.590144 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1316/2630] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:42 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:03:42.594664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c79665-f1cd-44b5-ae41-6775b4e9aa21 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:42.607079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c79665-f1cd-44b5-ae41-6775b4e9aa21 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:42.620288 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:42.683326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46470788-99e4-4469-851b-5ce2533710ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:42.684446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46470788-99e4-4469-851b-5ce2533710ad admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:42.684842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46470788-99e4-4469-851b-5ce2533710ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:42.685301 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46470788-99e4-4469-851b-5ce2533710ad admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:42.686126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46470788-99e4-4469-851b-5ce2533710ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:42.689045 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46470788-99e4-4469-851b-5ce2533710ad admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:42.766742 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1315/2631] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:42 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:42.771032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfeb3e-3539-47c9-bd38-49d88310cdc6 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:42.782982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfeb3e-3539-47c9-bd38-49d88310cdc6 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:42.788556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6937d2-85b1-4fde-bad4-52baad641bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:42.789487 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6937d2-85b1-4fde-bad4-52baad641bad admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:42.789992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6937d2-85b1-4fde-bad4-52baad641bad admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:42.790556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6937d2-85b1-4fde-bad4-52baad641bad admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:42.791615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6937d2-85b1-4fde-bad4-52baad641bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:42.794049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6937d2-85b1-4fde-bad4-52baad641bad admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:42.800582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:42.816526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1317/2632] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:42.820722 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46470788-99e4-4469-851b-5ce2533710ad admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:42.831997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46470788-99e4-4469-851b-5ce2533710ad admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:42.837376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd950b-8282-4a60-b37d-0f512ba399cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:42.838067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd950b-8282-4a60-b37d-0f512ba399cb admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:42.838423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd950b-8282-4a60-b37d-0f512ba399cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:42.838712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd950b-8282-4a60-b37d-0f512ba399cb admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:42.839487 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd950b-8282-4a60-b37d-0f512ba399cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:42.841412 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd950b-8282-4a60-b37d-0f512ba399cb admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:42.856442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1316/2633] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:42.860450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6937d2-85b1-4fde-bad4-52baad641bad admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:42.875190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6937d2-85b1-4fde-bad4-52baad641bad admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:42.880739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090206b-f581-4aeb-ad2e-db87bbb9d8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:42.882579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090206b-f581-4aeb-ad2e-db87bbb9d8dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:42.882865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090206b-f581-4aeb-ad2e-db87bbb9d8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:42.883150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090206b-f581-4aeb-ad2e-db87bbb9d8dc admin admin] PATH_INFO: `/v3/projects/27b30b23b08a497eb7ed308cd7805661/users/5edd0388db304db7aafc5c312bfecfa7/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:42.902180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090206b-f581-4aeb-ad2e-db87bbb9d8dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b30b23b08a497eb7ed308cd7805661, user_id=5edd0388db304db7aafc5c312bfecfa7, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:42.905414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090206b-f581-4aeb-ad2e-db87bbb9d8dc admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:43.024100 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1318/2634] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:42 2022] PUT /identity/v3/projects/27b30b23b08a497eb7ed308cd7805661/users/5edd0388db304db7aafc5c312bfecfa7/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:43.034931 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf0c55-8821-42a2-ab0f-3d0462a0d3a7 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:43.035245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf0c55-8821-42a2-ab0f-3d0462a0d3a7 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:43.035530 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf0c55-8821-42a2-ab0f-3d0462a0d3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:43.066185 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-3aaf0c55-8821-42a2-ab0f-3d0462a0d3a7 None None] MFA Rules not processed for user `5edd0388db304db7aafc5c312bfecfa7`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:43.067468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3aaf0c55-8821-42a2-ab0f-3d0462a0d3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:43.159672 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1317/2635] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:43.165051 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090206b-f581-4aeb-ad2e-db87bbb9d8dc admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:43.242657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090206b-f581-4aeb-ad2e-db87bbb9d8dc admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:43.250242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fea55-7d7e-412c-85da-6f0ed0e9c7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:43.251446 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fea55-7d7e-412c-85da-6f0ed0e9c7ae admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:43.252007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fea55-7d7e-412c-85da-6f0ed0e9c7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:43.252628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fea55-7d7e-412c-85da-6f0ed0e9c7ae admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:43.253945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fea55-7d7e-412c-85da-6f0ed0e9c7ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:43.257755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fea55-7d7e-412c-85da-6f0ed0e9c7ae admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:43.395941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1319/2636] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:43 2022] POST /identity/v3/projects => generated 377 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:03:43.400050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf0c55-8821-42a2-ab0f-3d0462a0d3a7 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:43.414029 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf0c55-8821-42a2-ab0f-3d0462a0d3a7 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:43.482699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ed1b7-7d3c-438a-ac5d-3245a825707d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:43.483651 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ed1b7-7d3c-438a-ac5d-3245a825707d admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:43.485102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ed1b7-7d3c-438a-ac5d-3245a825707d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:43.485745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ed1b7-7d3c-438a-ac5d-3245a825707d admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:43.486973 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ed1b7-7d3c-438a-ac5d-3245a825707d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:43.490822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ed1b7-7d3c-438a-ac5d-3245a825707d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:43.579156 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1318/2637] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:43 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:43.583079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fea55-7d7e-412c-85da-6f0ed0e9c7ae admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:43.594571 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fea55-7d7e-412c-85da-6f0ed0e9c7ae admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:43.599457 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcca4ca-7490-4829-aa86-3e9fd49e484b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:43.600132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcca4ca-7490-4829-aa86-3e9fd49e484b admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:43.601029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcca4ca-7490-4829-aa86-3e9fd49e484b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:43.601325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcca4ca-7490-4829-aa86-3e9fd49e484b admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:43.602160 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcca4ca-7490-4829-aa86-3e9fd49e484b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:43.603845 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcca4ca-7490-4829-aa86-3e9fd49e484b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:43.618788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1320/2638] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:43.622490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ed1b7-7d3c-438a-ac5d-3245a825707d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:43.634758 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ed1b7-7d3c-438a-ac5d-3245a825707d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:43.641311 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c43ae6-431b-4e65-bcca-8b817c9e8db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:43.642059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c43ae6-431b-4e65-bcca-8b817c9e8db2 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:43.642353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c43ae6-431b-4e65-bcca-8b817c9e8db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:43.642634 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c43ae6-431b-4e65-bcca-8b817c9e8db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:43.643379 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c43ae6-431b-4e65-bcca-8b817c9e8db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:43.645039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c43ae6-431b-4e65-bcca-8b817c9e8db2 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:43.653042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:43.658489 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1319/2639] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 16:03:43.663784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcca4ca-7490-4829-aa86-3e9fd49e484b admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:43.674683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcca4ca-7490-4829-aa86-3e9fd49e484b admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:43.679668 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c359c68-6fce-4673-971d-be90e8411c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:43.680642 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c359c68-6fce-4673-971d-be90e8411c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:43.680933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c359c68-6fce-4673-971d-be90e8411c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:43.681212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c359c68-6fce-4673-971d-be90e8411c05 admin admin] PATH_INFO: `/v3/projects/1c994e529edf4239b56e60c7381bf35d/users/5930ffbf0199409a97e13cae2d7c24fb/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:43.700830 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c359c68-6fce-4673-971d-be90e8411c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c994e529edf4239b56e60c7381bf35d, user_id=5930ffbf0199409a97e13cae2d7c24fb, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:43.703641 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c359c68-6fce-4673-971d-be90e8411c05 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:43.836832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1321/2640] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:43 2022] PUT /identity/v3/projects/1c994e529edf4239b56e60c7381bf35d/users/5930ffbf0199409a97e13cae2d7c24fb/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:43.849364 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42e1cb-21e3-4174-af4a-5ce33c2f8134 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:43.849956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42e1cb-21e3-4174-af4a-5ce33c2f8134 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:43.850550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42e1cb-21e3-4174-af4a-5ce33c2f8134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:43.915722 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-8a42e1cb-21e3-4174-af4a-5ce33c2f8134 None None] MFA Rules not processed for user `5930ffbf0199409a97e13cae2d7c24fb`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:43.918123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8a42e1cb-21e3-4174-af4a-5ce33c2f8134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:44.030569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1320/2641] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:44.035017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c359c68-6fce-4673-971d-be90e8411c05 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:44.121178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c359c68-6fce-4673-971d-be90e8411c05 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:44.125799 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aee5d6-1002-4424-8193-3d352a1ff262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:44.126466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aee5d6-1002-4424-8193-3d352a1ff262 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:44.126867 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aee5d6-1002-4424-8193-3d352a1ff262 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:44.127203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aee5d6-1002-4424-8193-3d352a1ff262 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:44.128042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aee5d6-1002-4424-8193-3d352a1ff262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:44.130162 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aee5d6-1002-4424-8193-3d352a1ff262 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:44.234910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1322/2642] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:44 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:44.239253 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42e1cb-21e3-4174-af4a-5ce33c2f8134 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:44.251922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42e1cb-21e3-4174-af4a-5ce33c2f8134 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:44.335730 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8576a1-04ee-45d3-8eee-1fa72196b3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:44.336930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8576a1-04ee-45d3-8eee-1fa72196b3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:44.337408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8576a1-04ee-45d3-8eee-1fa72196b3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:44.337893 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8576a1-04ee-45d3-8eee-1fa72196b3f8 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:44.339109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8576a1-04ee-45d3-8eee-1fa72196b3f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:44.342563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8576a1-04ee-45d3-8eee-1fa72196b3f8 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:44.457197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1321/2643] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:44 2022] POST /identity/v3/users => generated 295 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:44.458488 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aee5d6-1002-4424-8193-3d352a1ff262 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:44.470371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aee5d6-1002-4424-8193-3d352a1ff262 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:44.476527 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dac98c-fe20-4b90-8cc9-a996d75e66e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:44.477256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dac98c-fe20-4b90-8cc9-a996d75e66e9 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:44.478104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dac98c-fe20-4b90-8cc9-a996d75e66e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:44.478386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dac98c-fe20-4b90-8cc9-a996d75e66e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:44.479140 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dac98c-fe20-4b90-8cc9-a996d75e66e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:44.480804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dac98c-fe20-4b90-8cc9-a996d75e66e9 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:44.483444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:44.496032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1323/2644] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:44.501834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8576a1-04ee-45d3-8eee-1fa72196b3f8 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:44.513555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8576a1-04ee-45d3-8eee-1fa72196b3f8 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:44.518585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3ee43-135d-4721-8d46-d2c01c711719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:44.519600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3ee43-135d-4721-8d46-d2c01c711719 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:44.519895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3ee43-135d-4721-8d46-d2c01c711719 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:44.520178 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3ee43-135d-4721-8d46-d2c01c711719 admin admin] PATH_INFO: `/v3/projects/454f7265a6ab42b99f04fb8ecc6ddca5/users/0dd328ce8baa4617bd1553807105acf5/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:44.544152 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3ee43-135d-4721-8d46-d2c01c711719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454f7265a6ab42b99f04fb8ecc6ddca5, user_id=0dd328ce8baa4617bd1553807105acf5, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:44.547049 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3ee43-135d-4721-8d46-d2c01c711719 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:44.640830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1322/2645] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:44 2022] PUT /identity/v3/projects/454f7265a6ab42b99f04fb8ecc6ddca5/users/0dd328ce8baa4617bd1553807105acf5/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:44.644787 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dac98c-fe20-4b90-8cc9-a996d75e66e9 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:44.658853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dac98c-fe20-4b90-8cc9-a996d75e66e9 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:44.747972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e75ab-8d1f-4b12-9284-6a0bb5a0e707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:44.748692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e75ab-8d1f-4b12-9284-6a0bb5a0e707 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:44.748980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e75ab-8d1f-4b12-9284-6a0bb5a0e707 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:44.749266 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e75ab-8d1f-4b12-9284-6a0bb5a0e707 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:44.751654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e75ab-8d1f-4b12-9284-6a0bb5a0e707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:44.753334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e75ab-8d1f-4b12-9284-6a0bb5a0e707 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:44.765997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1324/2646] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:44.770242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3ee43-135d-4721-8d46-d2c01c711719 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:44.783289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3ee43-135d-4721-8d46-d2c01c711719 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:44.788952 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d503a-2f8a-42a5-ae6b-55f4cd24c0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:44.789889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d503a-2f8a-42a5-ae6b-55f4cd24c0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:44.790171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d503a-2f8a-42a5-ae6b-55f4cd24c0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:44.790537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d503a-2f8a-42a5-ae6b-55f4cd24c0a9 admin admin] PATH_INFO: `/v3/projects/454f7265a6ab42b99f04fb8ecc6ddca5/users/0dd328ce8baa4617bd1553807105acf5/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:44.792878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d503a-2f8a-42a5-ae6b-55f4cd24c0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454f7265a6ab42b99f04fb8ecc6ddca5, user_id=0dd328ce8baa4617bd1553807105acf5, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:44.797394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d503a-2f8a-42a5-ae6b-55f4cd24c0a9 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:44.810968 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1323/2647] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:44 2022] PUT /identity/v3/projects/454f7265a6ab42b99f04fb8ecc6ddca5/users/0dd328ce8baa4617bd1553807105acf5/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:44.823927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764833c-d134-4336-ab1d-8754a3c8e4bf None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:44.824217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764833c-d134-4336-ab1d-8754a3c8e4bf None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:44.824525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764833c-d134-4336-ab1d-8754a3c8e4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:44.857185 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e764833c-d134-4336-ab1d-8754a3c8e4bf None None] MFA Rules not processed for user `0dd328ce8baa4617bd1553807105acf5`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:44.858429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e764833c-d134-4336-ab1d-8754a3c8e4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:44.974646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1325/2648] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:03:44.982517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d503a-2f8a-42a5-ae6b-55f4cd24c0a9 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:44.984564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-578d503a-2f8a-42a5-ae6b-55f4cd24c0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:45.051827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:45.070602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d503a-2f8a-42a5-ae6b-55f4cd24c0a9 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:45.077170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50850702-a8df-4f27-830f-fbc5fe2a7adf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:45.078079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850702-a8df-4f27-830f-fbc5fe2a7adf service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:45.078340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850702-a8df-4f27-830f-fbc5fe2a7adf service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:45.078719 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50850702-a8df-4f27-830f-fbc5fe2a7adf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:45.096808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50850702-a8df-4f27-830f-fbc5fe2a7adf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:45.097984 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50850702-a8df-4f27-830f-fbc5fe2a7adf service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:45.124663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1324/2649] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:45.220423 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764833c-d134-4336-ab1d-8754a3c8e4bf None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:45.222395 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e764833c-d134-4336-ab1d-8754a3c8e4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:45.247477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764833c-d134-4336-ab1d-8754a3c8e4bf None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:45.253298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff0cbf-1fa2-4552-82cb-94db309395d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:45.254010 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0cbf-1fa2-4552-82cb-94db309395d5 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:45.254420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0cbf-1fa2-4552-82cb-94db309395d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:45.254936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff0cbf-1fa2-4552-82cb-94db309395d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:45.340069 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff0cbf-1fa2-4552-82cb-94db309395d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:45.341515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff0cbf-1fa2-4552-82cb-94db309395d5 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:45.352550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1326/2650] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:50.168927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50850702-a8df-4f27-830f-fbc5fe2a7adf service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:50.233956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50850702-a8df-4f27-830f-fbc5fe2a7adf service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:50.238828 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:50.239696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:50.239982 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:50.240262 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] PATH_INFO: `/v3/users/5edd0388db304db7aafc5c312bfecfa7` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:50.242059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5edd0388db304db7aafc5c312bfecfa7)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:50.243995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:50.386083 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5edd0388db304db7aafc5c312bfecfa7'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:50.386762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5edd0388db304db7aafc5c312bfecfa7'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:50.400281 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5edd0388db304db7aafc5c312bfecfa7'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:50.401034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5edd0388db304db7aafc5c312bfecfa7'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:50.408015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5edd0388db304db7aafc5c312bfecfa7'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:50.456956 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1325/2651] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:03:50 2022] DELETE /identity/v3/users/5edd0388db304db7aafc5c312bfecfa7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:50.461393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334680a-6a91-46af-b601-e714aade5914 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:50.461959 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334680a-6a91-46af-b601-e714aade5914 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:50.461959 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334680a-6a91-46af-b601-e714aade5914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:50.479882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-6334680a-6a91-46af-b601-e714aade5914 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:50.481204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6334680a-6a91-46af-b601-e714aade5914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:50.494704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:50.543454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1327/2652] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:50.553798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:50.555738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:50.639916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff7339-f20f-4e03-8358-4d31ea8c8d1d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:50.645249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb73593-1c34-4932-8e07-3b0536f60f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:50.646334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73593-1c34-4932-8e07-3b0536f60f08 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:50.646783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73593-1c34-4932-8e07-3b0536f60f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:50.647217 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb73593-1c34-4932-8e07-3b0536f60f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:50.661508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb73593-1c34-4932-8e07-3b0536f60f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:50.664453 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb73593-1c34-4932-8e07-3b0536f60f08 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:50.679564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1326/2653] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:51.156932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334680a-6a91-46af-b601-e714aade5914 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:51.163429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6334680a-6a91-46af-b601-e714aade5914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:51.172152 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334680a-6a91-46af-b601-e714aade5914 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:51.184347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:51.192121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:51.192121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:51.192121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] PATH_INFO: `/v3/projects/27b30b23b08a497eb7ed308cd7805661` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:51.192121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b30b23b08a497eb7ed308cd7805661)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:51.192121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:51.310842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b30b23b08a497eb7ed308cd7805661', 'request_id': 'req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:51.321952 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b30b23b08a497eb7ed308cd7805661', 'request_id': 'req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:51.330680 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b30b23b08a497eb7ed308cd7805661', 'request_id': 'req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:51.393491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] The token cache is being invalidate because project 27b30b23b08a497eb7ed308cd7805661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:03:51.393724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:51.394428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:51.396224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1328/2654] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:03:51 2022] DELETE /identity/v3/projects/27b30b23b08a497eb7ed308cd7805661 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:51.401028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb73593-1c34-4932-8e07-3b0536f60f08 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:51.411406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb73593-1c34-4932-8e07-3b0536f60f08 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:51.415154 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:51.471504 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:51.472428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:51.475213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:51.475213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] PATH_INFO: `/v3/users/5930ffbf0199409a97e13cae2d7c24fb` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:51.475213 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5930ffbf0199409a97e13cae2d7c24fb)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:51.476179 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:51.669556 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5930ffbf0199409a97e13cae2d7c24fb'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:51.670221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5930ffbf0199409a97e13cae2d7c24fb'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:51.683357 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5930ffbf0199409a97e13cae2d7c24fb'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:51.684053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5930ffbf0199409a97e13cae2d7c24fb'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:51.691059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5930ffbf0199409a97e13cae2d7c24fb'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:51.769238 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1327/2655] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:03:51 2022] DELETE /identity/v3/users/5930ffbf0199409a97e13cae2d7c24fb => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:52.190122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:52.191810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:52.209801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1f5ca-a38e-43f5-b7d7-8ca1051f0a84 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:52.274746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:52.275622 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:52.275909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:52.276190 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] PATH_INFO: `/v3/projects/1c994e529edf4239b56e60c7381bf35d` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:52.277469 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c994e529edf4239b56e60c7381bf35d)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:52.279521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:52.372898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:52.377003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c994e529edf4239b56e60c7381bf35d', 'request_id': 'req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:52.383607 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c994e529edf4239b56e60c7381bf35d', 'request_id': 'req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:52.389853 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c994e529edf4239b56e60c7381bf35d', 'request_id': 'req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:52.440349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] The token cache is being invalidate because project 1c994e529edf4239b56e60c7381bf35d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:03:52.440755 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:52.441750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:52.443732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1329/2656] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:03:52 2022] DELETE /identity/v3/projects/1c994e529edf4239b56e60c7381bf35d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:52.447809 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:52.449346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:52.467916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237d682-677f-4fc8-afd7-6c4baf4c97cb admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:52.529576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:52.530495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:52.530816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:52.531131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] PATH_INFO: `/v3/users/0dd328ce8baa4617bd1553807105acf5` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:52.532390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dd328ce8baa4617bd1553807105acf5)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:52.534601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:52.657153 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:52.771401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dd328ce8baa4617bd1553807105acf5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:52.772589 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dd328ce8baa4617bd1553807105acf5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:52.784477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dd328ce8baa4617bd1553807105acf5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:52.785136 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dd328ce8baa4617bd1553807105acf5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:52.792844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1bc6fd76-e3d3-44d0-99dd-386e7ed54bec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dd328ce8baa4617bd1553807105acf5'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:52.866172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1328/2657] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:03:52 2022] DELETE /identity/v3/users/0dd328ce8baa4617bd1553807105acf5 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:53.273595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:53.275176 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:53.297273 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e75ef0-043e-4a28-939e-b48e3d48d1b7 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:53.359470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:53.360422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:53.360692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:53.360972 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] PATH_INFO: `/v3/projects/454f7265a6ab42b99f04fb8ecc6ddca5` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:53.362173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=454f7265a6ab42b99f04fb8ecc6ddca5)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:53.364459 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:53.465616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454f7265a6ab42b99f04fb8ecc6ddca5', 'request_id': 'req-12996178-754d-458c-af19-21e0dadd1fac'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:53.471981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454f7265a6ab42b99f04fb8ecc6ddca5', 'request_id': 'req-12996178-754d-458c-af19-21e0dadd1fac'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:53.477991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454f7265a6ab42b99f04fb8ecc6ddca5', 'request_id': 'req-12996178-754d-458c-af19-21e0dadd1fac'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:53.548278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] The token cache is being invalidate because project 454f7265a6ab42b99f04fb8ecc6ddca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:03:53.548278 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:53.548896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-12996178-754d-458c-af19-21e0dadd1fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:53.550614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1330/2658] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:03:53 2022] DELETE /identity/v3/projects/454f7265a6ab42b99f04fb8ecc6ddca5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:53.557220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69677f-4102-4d07-8452-01c44574fc93 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:53.557967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69677f-4102-4d07-8452-01c44574fc93 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:53.557967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69677f-4102-4d07-8452-01c44574fc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:53.577663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ea69677f-4102-4d07-8452-01c44574fc93 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:53.578894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ea69677f-4102-4d07-8452-01c44574fc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:53.646590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1329/2659] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:03:53.654938 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0882c71-4e64-4991-8ddb-1364c119211a None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:53.655233 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0882c71-4e64-4991-8ddb-1364c119211a None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:53.655516 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0882c71-4e64-4991-8ddb-1364c119211a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:53.683213 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b0882c71-4e64-4991-8ddb-1364c119211a None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:53.684821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b0882c71-4e64-4991-8ddb-1364c119211a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:53.694748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1331/2660] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:53.700627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69677f-4102-4d07-8452-01c44574fc93 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:53.735503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69677f-4102-4d07-8452-01c44574fc93 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:53.735503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e03a8-d214-4b0e-96d5-49b0e4e1506b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:53.735503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e03a8-d214-4b0e-96d5-49b0e4e1506b admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:53.735503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e03a8-d214-4b0e-96d5-49b0e4e1506b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:53.736473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e03a8-d214-4b0e-96d5-49b0e4e1506b admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:53.736473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e03a8-d214-4b0e-96d5-49b0e4e1506b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:03:53.736473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e03a8-d214-4b0e-96d5-49b0e4e1506b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:53.736473 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e03a8-d214-4b0e-96d5-49b0e4e1506b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:53.736955 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1330/2661] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:03:53.744043 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0882c71-4e64-4991-8ddb-1364c119211a None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:53.756882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0882c71-4e64-4991-8ddb-1364c119211a None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:53.761790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa14541-382a-48e0-a1c4-eb91d54da8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:53.762468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa14541-382a-48e0-a1c4-eb91d54da8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:53.762758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa14541-382a-48e0-a1c4-eb91d54da8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:53.763045 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa14541-382a-48e0-a1c4-eb91d54da8e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:53.763797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa14541-382a-48e0-a1c4-eb91d54da8e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:53.765764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa14541-382a-48e0-a1c4-eb91d54da8e3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:54.536293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1332/2662] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:53 2022] POST /identity/v3/projects => generated 383 bytes in 793 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:54.540995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e03a8-d214-4b0e-96d5-49b0e4e1506b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:54.551171 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e03a8-d214-4b0e-96d5-49b0e4e1506b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:54.607601 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b12195-a70d-47c6-9332-537a99ec498f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:54.608424 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b12195-a70d-47c6-9332-537a99ec498f admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:54.608573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b12195-a70d-47c6-9332-537a99ec498f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:54.608800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b12195-a70d-47c6-9332-537a99ec498f admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:54.609501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b12195-a70d-47c6-9332-537a99ec498f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:54.611407 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b12195-a70d-47c6-9332-537a99ec498f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:54.720092 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1331/2663] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:54 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:03:54.724569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa14541-382a-48e0-a1c4-eb91d54da8e3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:54.734999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa14541-382a-48e0-a1c4-eb91d54da8e3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:54.739745 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b878dba-8c6c-4f91-83d0-48c41dac8888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:54.746479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b878dba-8c6c-4f91-83d0-48c41dac8888 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:54.746479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b878dba-8c6c-4f91-83d0-48c41dac8888 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:54.746479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b878dba-8c6c-4f91-83d0-48c41dac8888 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:54.746479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b878dba-8c6c-4f91-83d0-48c41dac8888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:54.746479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b878dba-8c6c-4f91-83d0-48c41dac8888 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:54.754621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1333/2664] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:54.759349 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b12195-a70d-47c6-9332-537a99ec498f admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:54.771149 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b12195-a70d-47c6-9332-537a99ec498f admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:54.776536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175176f4-2d9b-46d7-84f1-11c7b91609ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:54.777225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175176f4-2d9b-46d7-84f1-11c7b91609ba admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:54.777512 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175176f4-2d9b-46d7-84f1-11c7b91609ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:54.777790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175176f4-2d9b-46d7-84f1-11c7b91609ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:54.780877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175176f4-2d9b-46d7-84f1-11c7b91609ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:54.780877 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175176f4-2d9b-46d7-84f1-11c7b91609ba admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:54.794429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1332/2665] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:54.795691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b878dba-8c6c-4f91-83d0-48c41dac8888 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:54.813763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b878dba-8c6c-4f91-83d0-48c41dac8888 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:54.821579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00a366-2028-45dc-b879-3137b2e32be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:54.823103 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00a366-2028-45dc-b879-3137b2e32be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:54.823615 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00a366-2028-45dc-b879-3137b2e32be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:54.824142 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00a366-2028-45dc-b879-3137b2e32be1 admin admin] PATH_INFO: `/v3/projects/e6949e43cc1f47b6ba42855be986599e/users/699373129c1244cea312e43c1efc033a/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:54.849882 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00a366-2028-45dc-b879-3137b2e32be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6949e43cc1f47b6ba42855be986599e, user_id=699373129c1244cea312e43c1efc033a, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:54.855348 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00a366-2028-45dc-b879-3137b2e32be1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:54.960576 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:54.966500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1334/2666] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:54 2022] PUT /identity/v3/projects/e6949e43cc1f47b6ba42855be986599e/users/699373129c1244cea312e43c1efc033a/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:54.980970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1e56-23e3-47d4-ac11-683135fdcd4f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:54.981263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1e56-23e3-47d4-ac11-683135fdcd4f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:54.981543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1e56-23e3-47d4-ac11-683135fdcd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:55.001671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:55.009738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6f1e1e56-23e3-47d4-ac11-683135fdcd4f None None] MFA Rules not processed for user `699373129c1244cea312e43c1efc033a`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:55.010943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6f1e1e56-23e3-47d4-ac11-683135fdcd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:55.074124 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1333/2667] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:55.078860 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00a366-2028-45dc-b879-3137b2e32be1 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:55.140020 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00a366-2028-45dc-b879-3137b2e32be1 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:55.144717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9732d4-d76e-4d94-b47f-359249ef6373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:55.145377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9732d4-d76e-4d94-b47f-359249ef6373 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:55.145660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9732d4-d76e-4d94-b47f-359249ef6373 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:55.145934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9732d4-d76e-4d94-b47f-359249ef6373 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:55.146686 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9732d4-d76e-4d94-b47f-359249ef6373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:55.148602 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9732d4-d76e-4d94-b47f-359249ef6373 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:55.233856 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1335/2668] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:55 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:55.238350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1e56-23e3-47d4-ac11-683135fdcd4f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:55.249761 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1e56-23e3-47d4-ac11-683135fdcd4f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:55.310878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737db349-a39a-4703-8240-9e8db74b15c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:55.311564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737db349-a39a-4703-8240-9e8db74b15c7 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:55.311848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737db349-a39a-4703-8240-9e8db74b15c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:55.312127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737db349-a39a-4703-8240-9e8db74b15c7 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:55.312883 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737db349-a39a-4703-8240-9e8db74b15c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:55.314827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737db349-a39a-4703-8240-9e8db74b15c7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:55.392803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1334/2669] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:55 2022] POST /identity/v3/users => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:55.396217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9732d4-d76e-4d94-b47f-359249ef6373 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:55.406577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9732d4-d76e-4d94-b47f-359249ef6373 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:55.411247 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6815d0-3c8e-4bf6-a85d-b5c68136650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:55.411907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6815d0-3c8e-4bf6-a85d-b5c68136650e admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:55.412719 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6815d0-3c8e-4bf6-a85d-b5c68136650e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:55.413004 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6815d0-3c8e-4bf6-a85d-b5c68136650e admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:55.413721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6815d0-3c8e-4bf6-a85d-b5c68136650e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:55.415344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6815d0-3c8e-4bf6-a85d-b5c68136650e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:55.426253 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1336/2670] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:55.431250 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737db349-a39a-4703-8240-9e8db74b15c7 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:55.441065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737db349-a39a-4703-8240-9e8db74b15c7 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:55.445743 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80240856-5694-4acd-9235-74020288a282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:55.446618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80240856-5694-4acd-9235-74020288a282 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:55.446902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80240856-5694-4acd-9235-74020288a282 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:55.447180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80240856-5694-4acd-9235-74020288a282 admin admin] PATH_INFO: `/v3/projects/7568a012ef7c42df9bcdbaa96b9bb81b/users/84a1db96ba974f8bacc263576bcb275d/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:55.463926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80240856-5694-4acd-9235-74020288a282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7568a012ef7c42df9bcdbaa96b9bb81b, user_id=84a1db96ba974f8bacc263576bcb275d, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:55.466581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80240856-5694-4acd-9235-74020288a282 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:55.554629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1335/2671] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:55 2022] PUT /identity/v3/projects/7568a012ef7c42df9bcdbaa96b9bb81b/users/84a1db96ba974f8bacc263576bcb275d/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:03:55.558608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6815d0-3c8e-4bf6-a85d-b5c68136650e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:55.569764 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6815d0-3c8e-4bf6-a85d-b5c68136650e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:55.625594 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10cce6-7df8-4705-b844-b3058b7aa81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:55.626264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10cce6-7df8-4705-b844-b3058b7aa81c admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:55.626548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10cce6-7df8-4705-b844-b3058b7aa81c admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:55.626823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10cce6-7df8-4705-b844-b3058b7aa81c admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:55.627540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10cce6-7df8-4705-b844-b3058b7aa81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:55.629175 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10cce6-7df8-4705-b844-b3058b7aa81c admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:55.638771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1337/2672] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:55.643142 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80240856-5694-4acd-9235-74020288a282 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:55.653953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80240856-5694-4acd-9235-74020288a282 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:55.658631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08bbec-ca8b-482a-b9a6-fe01bf7213f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:55.659514 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08bbec-ca8b-482a-b9a6-fe01bf7213f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:55.659801 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08bbec-ca8b-482a-b9a6-fe01bf7213f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:55.660082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08bbec-ca8b-482a-b9a6-fe01bf7213f6 admin admin] PATH_INFO: `/v3/projects/7568a012ef7c42df9bcdbaa96b9bb81b/users/84a1db96ba974f8bacc263576bcb275d/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:55.662095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08bbec-ca8b-482a-b9a6-fe01bf7213f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7568a012ef7c42df9bcdbaa96b9bb81b, user_id=84a1db96ba974f8bacc263576bcb275d, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:55.665363 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08bbec-ca8b-482a-b9a6-fe01bf7213f6 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:55.675655 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1336/2673] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:55 2022] PUT /identity/v3/projects/7568a012ef7c42df9bcdbaa96b9bb81b/users/84a1db96ba974f8bacc263576bcb275d/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:55.685859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5317e-43f8-4013-b838-74dc53b5ae3d None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:55.686186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5317e-43f8-4013-b838-74dc53b5ae3d None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:55.686474 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5317e-43f8-4013-b838-74dc53b5ae3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:55.712881 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-dbc5317e-43f8-4013-b838-74dc53b5ae3d None None] MFA Rules not processed for user `84a1db96ba974f8bacc263576bcb275d`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:55.714099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-dbc5317e-43f8-4013-b838-74dc53b5ae3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:55.773720 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:55.783498 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1338/2674] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:55.795965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08bbec-ca8b-482a-b9a6-fe01bf7213f6 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:55.797757 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7f08bbec-ca8b-482a-b9a6-fe01bf7213f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:55.854966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08bbec-ca8b-482a-b9a6-fe01bf7213f6 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:55.858689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b4d3b-2809-43dc-900d-76ffe69a2adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:55.859340 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b4d3b-2809-43dc-900d-76ffe69a2adb service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:55.859627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b4d3b-2809-43dc-900d-76ffe69a2adb service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:55.859908 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b4d3b-2809-43dc-900d-76ffe69a2adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:55.868474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:55.871392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b4d3b-2809-43dc-900d-76ffe69a2adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:55.872988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b4d3b-2809-43dc-900d-76ffe69a2adb service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:55.883224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1337/2675] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:56.061618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5317e-43f8-4013-b838-74dc53b5ae3d None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:56.071819 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5317e-43f8-4013-b838-74dc53b5ae3d None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:56.075498 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e036080-ada1-4317-adfc-e35ae7b7fcf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:56.076145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e036080-ada1-4317-adfc-e35ae7b7fcf9 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:56.076466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e036080-ada1-4317-adfc-e35ae7b7fcf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:56.076735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e036080-ada1-4317-adfc-e35ae7b7fcf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:56.135282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e036080-ada1-4317-adfc-e35ae7b7fcf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:56.136758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e036080-ada1-4317-adfc-e35ae7b7fcf9 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:56.147324 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1339/2676] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:56.672174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b4d3b-2809-43dc-900d-76ffe69a2adb service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:56.733732 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b4d3b-2809-43dc-900d-76ffe69a2adb service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:56.738492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:56.739360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:56.739645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:56.739924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] PATH_INFO: `/v3/users/699373129c1244cea312e43c1efc033a` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:56.741207 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=699373129c1244cea312e43c1efc033a)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:56.743127 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:56.908066 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '699373129c1244cea312e43c1efc033a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:56.908678 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '699373129c1244cea312e43c1efc033a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:56.920179 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '699373129c1244cea312e43c1efc033a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:56.920832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '699373129c1244cea312e43c1efc033a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:56.928305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '699373129c1244cea312e43c1efc033a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:57.019780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1338/2677] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:03:56 2022] DELETE /identity/v3/users/699373129c1244cea312e43c1efc033a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:57.025059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc87f81-0229-4415-96a6-bad9eb3a25a8 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:57.025554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc87f81-0229-4415-96a6-bad9eb3a25a8 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:57.025643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc87f81-0229-4415-96a6-bad9eb3a25a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:57.043566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-6fc87f81-0229-4415-96a6-bad9eb3a25a8 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:57.044825 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6fc87f81-0229-4415-96a6-bad9eb3a25a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:57.104757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1340/2678] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:03:57.114720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:57.116630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:57.174218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c818cbf-cb0c-4456-8cda-50aec31b0328 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:57.178417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef697e-4223-4a78-a176-285b73822796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:57.179150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef697e-4223-4a78-a176-285b73822796 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:57.179524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef697e-4223-4a78-a176-285b73822796 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:57.179870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef697e-4223-4a78-a176-285b73822796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:57.189396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef697e-4223-4a78-a176-285b73822796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:57.190917 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef697e-4223-4a78-a176-285b73822796 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:57.201255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1339/2679] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:03:57.650274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc87f81-0229-4415-96a6-bad9eb3a25a8 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:57.654893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6fc87f81-0229-4415-96a6-bad9eb3a25a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:57.665131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc87f81-0229-4415-96a6-bad9eb3a25a8 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:57.669985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:57.670858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:57.671137 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:57.671435 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] PATH_INFO: `/v3/projects/e6949e43cc1f47b6ba42855be986599e` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:57.672679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6949e43cc1f47b6ba42855be986599e)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:57.674662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:57.774330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6949e43cc1f47b6ba42855be986599e', 'request_id': 'req-df050402-b974-4ccd-b33a-ce1bc03acaff'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:57.780448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6949e43cc1f47b6ba42855be986599e', 'request_id': 'req-df050402-b974-4ccd-b33a-ce1bc03acaff'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:57.785922 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6949e43cc1f47b6ba42855be986599e', 'request_id': 'req-df050402-b974-4ccd-b33a-ce1bc03acaff'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:57.865057 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] The token cache is being invalidate because project e6949e43cc1f47b6ba42855be986599e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:03:57.865345 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:57.865999 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:57.867597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1341/2680] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:03:57 2022] DELETE /identity/v3/projects/e6949e43cc1f47b6ba42855be986599e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:57.871549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef697e-4223-4a78-a176-285b73822796 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:57.881323 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef697e-4223-4a78-a176-285b73822796 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:57.893558 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:57.938043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:57.938926 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:57.939214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:57.939497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] PATH_INFO: `/v3/users/84a1db96ba974f8bacc263576bcb275d` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:57.940766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a1db96ba974f8bacc263576bcb275d)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:57.942691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:58.173233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a1db96ba974f8bacc263576bcb275d'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:58.173807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a1db96ba974f8bacc263576bcb275d'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:58.185862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a1db96ba974f8bacc263576bcb275d'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:58.186585 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a1db96ba974f8bacc263576bcb275d'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:58.193976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ff397f5-bb54-48d3-b5b3-4bb06765656e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a1db96ba974f8bacc263576bcb275d'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:58.256785 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1340/2681] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:03:57 2022] DELETE /identity/v3/users/84a1db96ba974f8bacc263576bcb275d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:03:58.703795 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:58.705872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:58.723371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df050402-b974-4ccd-b33a-ce1bc03acaff admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:58.737386 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:58.784455 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:58.785371 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:58.785654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:58.785929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] PATH_INFO: `/v3/projects/7568a012ef7c42df9bcdbaa96b9bb81b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:58.787753 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7568a012ef7c42df9bcdbaa96b9bb81b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:58.789681 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:58.912791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7568a012ef7c42df9bcdbaa96b9bb81b', 'request_id': 'req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:58.923120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7568a012ef7c42df9bcdbaa96b9bb81b', 'request_id': 'req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:58.931805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7568a012ef7c42df9bcdbaa96b9bb81b', 'request_id': 'req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:59.054214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] The token cache is being invalidate because project 7568a012ef7c42df9bcdbaa96b9bb81b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:03:59.054531 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:59.055181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-395cb7a4-dfdf-4a38-b5fc-1676f40b8ed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:03:59.056763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1342/2682] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:03:58 2022] DELETE /identity/v3/projects/7568a012ef7c42df9bcdbaa96b9bb81b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:59.061749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149027b-22f2-4d88-82f9-444806d8922b None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:59.062286 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149027b-22f2-4d88-82f9-444806d8922b None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:59.062386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149027b-22f2-4d88-82f9-444806d8922b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:59.083902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-2149027b-22f2-4d88-82f9-444806d8922b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:59.085751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2149027b-22f2-4d88-82f9-444806d8922b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:59.148351 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1341/2683] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:59.155844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c725b-a332-4fc8-b3d9-0a5f6c0f5920 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:59.156139 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c725b-a332-4fc8-b3d9-0a5f6c0f5920 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:59.156448 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c725b-a332-4fc8-b3d9-0a5f6c0f5920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:59.174913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-3f4c725b-a332-4fc8-b3d9-0a5f6c0f5920 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:59.176132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3f4c725b-a332-4fc8-b3d9-0a5f6c0f5920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:59.184885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1343/2684] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:59.189755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149027b-22f2-4d88-82f9-444806d8922b None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:59.200280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149027b-22f2-4d88-82f9-444806d8922b None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:59.205061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae6217-3ad3-4849-876a-a5e7263bb26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:59.205730 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae6217-3ad3-4849-876a-a5e7263bb26e admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:59.206019 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae6217-3ad3-4849-876a-a5e7263bb26e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:59.206300 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae6217-3ad3-4849-876a-a5e7263bb26e admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:59.207032 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae6217-3ad3-4849-876a-a5e7263bb26e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:03:59.207343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae6217-3ad3-4849-876a-a5e7263bb26e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:59.209024 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae6217-3ad3-4849-876a-a5e7263bb26e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:59.221040 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1342/2685] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:03:59.225444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c725b-a332-4fc8-b3d9-0a5f6c0f5920 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:59.234989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c725b-a332-4fc8-b3d9-0a5f6c0f5920 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:59.239948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a8f72-b3db-439c-a444-16344bd8c046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:59.241181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a8f72-b3db-439c-a444-16344bd8c046 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:59.241522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a8f72-b3db-439c-a444-16344bd8c046 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:59.241802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a8f72-b3db-439c-a444-16344bd8c046 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:59.242509 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a8f72-b3db-439c-a444-16344bd8c046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:59.244432 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a8f72-b3db-439c-a444-16344bd8c046 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:59.349895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1344/2686] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:59 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:03:59.353928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae6217-3ad3-4849-876a-a5e7263bb26e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:59.364716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae6217-3ad3-4849-876a-a5e7263bb26e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:59.387437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:59.424975 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904764d4-8b12-4ecf-86b9-4bbc4ea85f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:59.425667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904764d4-8b12-4ecf-86b9-4bbc4ea85f35 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:59.425951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904764d4-8b12-4ecf-86b9-4bbc4ea85f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:59.426228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904764d4-8b12-4ecf-86b9-4bbc4ea85f35 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:59.426923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904764d4-8b12-4ecf-86b9-4bbc4ea85f35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:59.428862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904764d4-8b12-4ecf-86b9-4bbc4ea85f35 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:59.506173 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1343/2687] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:59 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:59.510301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a8f72-b3db-439c-a444-16344bd8c046 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:59.520705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a8f72-b3db-439c-a444-16344bd8c046 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:59.525819 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca31ed-cb27-47ef-bc39-956dcaec274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:59.526484 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca31ed-cb27-47ef-bc39-956dcaec274d admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:59.526767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca31ed-cb27-47ef-bc39-956dcaec274d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:59.527047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca31ed-cb27-47ef-bc39-956dcaec274d admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:59.527766 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca31ed-cb27-47ef-bc39-956dcaec274d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:59.529391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca31ed-cb27-47ef-bc39-956dcaec274d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:59.540714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1345/2688] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:59.545012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904764d4-8b12-4ecf-86b9-4bbc4ea85f35 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:59.555101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904764d4-8b12-4ecf-86b9-4bbc4ea85f35 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:59.559966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ac46e-9035-4571-9ec4-1b7a83ea69ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:59.560671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ac46e-9035-4571-9ec4-1b7a83ea69ed admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:59.560988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ac46e-9035-4571-9ec4-1b7a83ea69ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:59.561242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ac46e-9035-4571-9ec4-1b7a83ea69ed admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:59.562005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ac46e-9035-4571-9ec4-1b7a83ea69ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:59.563604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ac46e-9035-4571-9ec4-1b7a83ea69ed admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:59.574649 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1344/2689] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:03:59.579567 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca31ed-cb27-47ef-bc39-956dcaec274d admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:59.590464 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca31ed-cb27-47ef-bc39-956dcaec274d admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:59.595146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4a14e-8c3e-42c2-926b-172926317613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:59.596028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a14e-8c3e-42c2-926b-172926317613 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:59.596311 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a14e-8c3e-42c2-926b-172926317613 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:59.596665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a14e-8c3e-42c2-926b-172926317613 admin admin] PATH_INFO: `/v3/projects/997868d1f33e4701adf03352ee4303c1/users/15bcfd38b5ce4d8f8a4b90e4a2170ee1/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:59.611135 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4a14e-8c3e-42c2-926b-172926317613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=997868d1f33e4701adf03352ee4303c1, user_id=15bcfd38b5ce4d8f8a4b90e4a2170ee1, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:59.614069 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4a14e-8c3e-42c2-926b-172926317613 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:59.683325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1346/2690] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:03:59 2022] PUT /identity/v3/projects/997868d1f33e4701adf03352ee4303c1/users/15bcfd38b5ce4d8f8a4b90e4a2170ee1/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:03:59.694717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c82b82-419a-4fd9-8238-7ce7a67fd4ea None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:59.695014 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c82b82-419a-4fd9-8238-7ce7a67fd4ea None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:59.695299 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c82b82-419a-4fd9-8238-7ce7a67fd4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:59.722480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-f5c82b82-419a-4fd9-8238-7ce7a67fd4ea None None] MFA Rules not processed for user `15bcfd38b5ce4d8f8a4b90e4a2170ee1`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:03:59.723780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f5c82b82-419a-4fd9-8238-7ce7a67fd4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:03:59.788986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1345/2691] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:03:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:03:59.793494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4a14e-8c3e-42c2-926b-172926317613 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:59.840729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:03:59.859699 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4a14e-8c3e-42c2-926b-172926317613 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:03:59.862515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55de130-d66d-459f-810d-30625390bb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:03:59.863471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55de130-d66d-459f-810d-30625390bb22 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:03:59.863757 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55de130-d66d-459f-810d-30625390bb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:03:59.864037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55de130-d66d-459f-810d-30625390bb22 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:03:59.864861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55de130-d66d-459f-810d-30625390bb22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:03:59.866884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55de130-d66d-459f-810d-30625390bb22 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:03:59.977619 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1347/2692] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:03:59 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:03:59.981690 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c82b82-419a-4fd9-8238-7ce7a67fd4ea None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:03:59.992902 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c82b82-419a-4fd9-8238-7ce7a67fd4ea None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:00.047571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4162e5-d9a2-4bc4-a2b9-e8036fe047d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:00.048249 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4162e5-d9a2-4bc4-a2b9-e8036fe047d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:00.048565 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4162e5-d9a2-4bc4-a2b9-e8036fe047d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:00.048849 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4162e5-d9a2-4bc4-a2b9-e8036fe047d0 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:00.049550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4162e5-d9a2-4bc4-a2b9-e8036fe047d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:00.051468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4162e5-d9a2-4bc4-a2b9-e8036fe047d0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:00.159711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1346/2693] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:03:59 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:04:00.163660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55de130-d66d-459f-810d-30625390bb22 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:00.174211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55de130-d66d-459f-810d-30625390bb22 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:00.179344 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc70fc-af25-4eb3-ba7e-f9afeaf193a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:00.180004 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc70fc-af25-4eb3-ba7e-f9afeaf193a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:00.180293 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc70fc-af25-4eb3-ba7e-f9afeaf193a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:00.180677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc70fc-af25-4eb3-ba7e-f9afeaf193a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:00.181416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc70fc-af25-4eb3-ba7e-f9afeaf193a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:00.183515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc70fc-af25-4eb3-ba7e-f9afeaf193a1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:00.195164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1348/2694] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:04:00.199229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4162e5-d9a2-4bc4-a2b9-e8036fe047d0 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:00.209328 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4162e5-d9a2-4bc4-a2b9-e8036fe047d0 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:00.213862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9b25-a014-4644-bf91-79a875677402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:00.214740 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9b25-a014-4644-bf91-79a875677402 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:00.215022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9b25-a014-4644-bf91-79a875677402 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:00.215298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9b25-a014-4644-bf91-79a875677402 admin admin] PATH_INFO: `/v3/projects/079b28c805454d03b8969138df67321c/users/7bcbc57843bd442aabfe77913457554b/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:00.229912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e9b25-a014-4644-bf91-79a875677402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079b28c805454d03b8969138df67321c, user_id=7bcbc57843bd442aabfe77913457554b, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:00.232633 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e9b25-a014-4644-bf91-79a875677402 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:00.314482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:00.320714 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1347/2695] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:04:00 2022] PUT /identity/v3/projects/079b28c805454d03b8969138df67321c/users/7bcbc57843bd442aabfe77913457554b/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:04:00.324704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc70fc-af25-4eb3-ba7e-f9afeaf193a1 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:00.334677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc70fc-af25-4eb3-ba7e-f9afeaf193a1 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:00.390963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a040579-889b-4cef-928c-72ca7492cde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:00.391662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a040579-889b-4cef-928c-72ca7492cde2 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:00.391949 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a040579-889b-4cef-928c-72ca7492cde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:00.392231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a040579-889b-4cef-928c-72ca7492cde2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:00.392997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a040579-889b-4cef-928c-72ca7492cde2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:00.394590 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a040579-889b-4cef-928c-72ca7492cde2 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:00.404186 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1349/2696] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:04:00.408315 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9b25-a014-4644-bf91-79a875677402 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:00.418985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9b25-a014-4644-bf91-79a875677402 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:00.423594 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af482d36-cf77-4dc5-9fe0-e223ea5af90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:00.424508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af482d36-cf77-4dc5-9fe0-e223ea5af90f admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:00.424791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af482d36-cf77-4dc5-9fe0-e223ea5af90f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:00.425061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af482d36-cf77-4dc5-9fe0-e223ea5af90f admin admin] PATH_INFO: `/v3/projects/079b28c805454d03b8969138df67321c/users/7bcbc57843bd442aabfe77913457554b/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:00.427002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af482d36-cf77-4dc5-9fe0-e223ea5af90f admin admin] RBAC: Authorizing `identity:create_grant(project_id=079b28c805454d03b8969138df67321c, user_id=7bcbc57843bd442aabfe77913457554b, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:00.430253 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af482d36-cf77-4dc5-9fe0-e223ea5af90f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:00.440137 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1348/2697] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:04:00 2022] PUT /identity/v3/projects/079b28c805454d03b8969138df67321c/users/7bcbc57843bd442aabfe77913457554b/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:00.449948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224feb1-33b5-464a-ab3c-079536d96e76 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:00.450242 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224feb1-33b5-464a-ab3c-079536d96e76 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:00.450529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224feb1-33b5-464a-ab3c-079536d96e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:00.476550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a224feb1-33b5-464a-ab3c-079536d96e76 None None] MFA Rules not processed for user `7bcbc57843bd442aabfe77913457554b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:04:00.482586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a224feb1-33b5-464a-ab3c-079536d96e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:00.543804 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1350/2698] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:04:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:04:00.554800 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af482d36-cf77-4dc5-9fe0-e223ea5af90f admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:00.556494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-af482d36-cf77-4dc5-9fe0-e223ea5af90f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:00.614860 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af482d36-cf77-4dc5-9fe0-e223ea5af90f admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:00.619247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dbd8a-a6b2-4f97-aef9-d51632427ab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:00.619910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dbd8a-a6b2-4f97-aef9-d51632427ab2 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:00.620198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dbd8a-a6b2-4f97-aef9-d51632427ab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:00.620531 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dbd8a-a6b2-4f97-aef9-d51632427ab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:00.680287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759dbd8a-a6b2-4f97-aef9-d51632427ab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:00.681694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759dbd8a-a6b2-4f97-aef9-d51632427ab2 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:00.691889 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1349/2699] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:04:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:04:02.913899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df44c1b-da7b-44ee-b827-77196a564152 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:02.918529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df44c1b-da7b-44ee-b827-77196a564152 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:02.918529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df44c1b-da7b-44ee-b827-77196a564152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:02.933948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-9df44c1b-da7b-44ee-b827-77196a564152 None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:04:02.935238 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-9df44c1b-da7b-44ee-b827-77196a564152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:02.968315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:02.997088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1351/2700] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 23 16:04:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:04:03.010012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dbd8a-a6b2-4f97-aef9-d51632427ab2 service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:03.011549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-759dbd8a-a6b2-4f97-aef9-d51632427ab2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:03.085321 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dbd8a-a6b2-4f97-aef9-d51632427ab2 service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:03.090280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb6cbb-465e-499c-af45-eba430c6559e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:03.090930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6cbb-465e-499c-af45-eba430c6559e service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:03.091216 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6cbb-465e-499c-af45-eba430c6559e service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:03.091495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6cbb-465e-499c-af45-eba430c6559e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:03.099614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb6cbb-465e-499c-af45-eba430c6559e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:03.101296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb6cbb-465e-499c-af45-eba430c6559e service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:03.109832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1350/2701] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 16:04:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 16:04:05.771850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df44c1b-da7b-44ee-b827-77196a564152 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:05.773252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-9df44c1b-da7b-44ee-b827-77196a564152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:05.785383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df44c1b-da7b-44ee-b827-77196a564152 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:05.789046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbba49b-ef19-4826-bab9-94bbf3ca95c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:05.789816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba49b-ef19-4826-bab9-94bbf3ca95c8 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:05.790157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba49b-ef19-4826-bab9-94bbf3ca95c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:05.790515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba49b-ef19-4826-bab9-94bbf3ca95c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:05.801807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbba49b-ef19-4826-bab9-94bbf3ca95c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:05.803572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbba49b-ef19-4826-bab9-94bbf3ca95c8 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:05.813333 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1352/2702] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:04:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:04:11.488392 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb6cbb-465e-499c-af45-eba430c6559e service swift] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:11.519214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:11.549703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb6cbb-465e-499c-af45-eba430c6559e service swift] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:11.554263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:11.555139 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:11.557579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:11.557579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] PATH_INFO: `/v3/users/15bcfd38b5ce4d8f8a4b90e4a2170ee1` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:11.557579 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15bcfd38b5ce4d8f8a4b90e4a2170ee1)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:11.559464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:11.725233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15bcfd38b5ce4d8f8a4b90e4a2170ee1'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:11.725884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15bcfd38b5ce4d8f8a4b90e4a2170ee1'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:11.738272 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15bcfd38b5ce4d8f8a4b90e4a2170ee1'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:11.738904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15bcfd38b5ce4d8f8a4b90e4a2170ee1'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:11.746422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15bcfd38b5ce4d8f8a4b90e4a2170ee1'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:11.836097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1351/2703] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:04:11 2022] DELETE /identity/v3/users/15bcfd38b5ce4d8f8a4b90e4a2170ee1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:11.841112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e8313-66bb-4f61-b6b3-7c9cc08f515f None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:11.841582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e8313-66bb-4f61-b6b3-7c9cc08f515f None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:11.841723 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e8313-66bb-4f61-b6b3-7c9cc08f515f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:11.859541 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-708e8313-66bb-4f61-b6b3-7c9cc08f515f None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:04:11.860765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-708e8313-66bb-4f61-b6b3-7c9cc08f515f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:11.920442 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1353/2704] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:04:11.930312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:11.931939 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:11.992970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203c6cb-8e59-44cf-ac4a-f175f64edfb5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:11.997544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de380424-5fb9-4d4c-8b8b-40a3e843f4d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:11.998241 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de380424-5fb9-4d4c-8b8b-40a3e843f4d2 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:11.998528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de380424-5fb9-4d4c-8b8b-40a3e843f4d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:11.998830 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de380424-5fb9-4d4c-8b8b-40a3e843f4d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:12.008409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de380424-5fb9-4d4c-8b8b-40a3e843f4d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:12.009895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de380424-5fb9-4d4c-8b8b-40a3e843f4d2 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:12.023242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1352/2705] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 16:04:12.406633 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e8313-66bb-4f61-b6b3-7c9cc08f515f None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:12.408343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-708e8313-66bb-4f61-b6b3-7c9cc08f515f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:12.429678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e8313-66bb-4f61-b6b3-7c9cc08f515f None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:12.435977 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:12.436257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:12.436595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:12.436880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] PATH_INFO: `/v3/projects/997868d1f33e4701adf03352ee4303c1` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:12.438095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] RBAC: Authorizing `identity:delete_project(project_id=997868d1f33e4701adf03352ee4303c1)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:12.440033 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:12.563092 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '997868d1f33e4701adf03352ee4303c1', 'request_id': 'req-74217a60-f51e-4336-aecc-b2e90acfebef'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:12.569377 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '997868d1f33e4701adf03352ee4303c1', 'request_id': 'req-74217a60-f51e-4336-aecc-b2e90acfebef'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:12.575788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '997868d1f33e4701adf03352ee4303c1', 'request_id': 'req-74217a60-f51e-4336-aecc-b2e90acfebef'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:12.647263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] The token cache is being invalidate because project 997868d1f33e4701adf03352ee4303c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:04:12.647554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:12.648182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:12.649852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1354/2706] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:04:12 2022] DELETE /identity/v3/projects/997868d1f33e4701adf03352ee4303c1 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:12.653972 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de380424-5fb9-4d4c-8b8b-40a3e843f4d2 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:12.663543 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de380424-5fb9-4d4c-8b8b-40a3e843f4d2 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:12.710995 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:12.720606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:12.727018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:12.727018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:12.727018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] PATH_INFO: `/v3/users/7bcbc57843bd442aabfe77913457554b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:12.727018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bcbc57843bd442aabfe77913457554b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:12.727018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:12.908575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:12.910034 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:12.964500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:12.982006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74217a60-f51e-4336-aecc-b2e90acfebef admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:12.987539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c53205-a4fd-434c-a0d3-b64696574a4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:12.988193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bcbc57843bd442aabfe77913457554b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:12.988316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c53205-a4fd-434c-a0d3-b64696574a4f service placement] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:12.988535 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c53205-a4fd-434c-a0d3-b64696574a4f service placement] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:12.988817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c53205-a4fd-434c-a0d3-b64696574a4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:12.989475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bcbc57843bd442aabfe77913457554b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:12.990676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e9c53205-a4fd-434c-a0d3-b64696574a4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:13.001986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bcbc57843bd442aabfe77913457554b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:13.002924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bcbc57843bd442aabfe77913457554b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:13.003070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c53205-a4fd-434c-a0d3-b64696574a4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:13.006295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c53205-a4fd-434c-a0d3-b64696574a4f service placement] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:13.009676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-25b0c649-4db0-4fec-9890-763bff3bb6f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bcbc57843bd442aabfe77913457554b'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:13.015269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1355/2707] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:04:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 16:04:13.128065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1353/2708] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:04:12 2022] DELETE /identity/v3/users/7bcbc57843bd442aabfe77913457554b => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:13.496927 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c53205-a4fd-434c-a0d3-b64696574a4f service placement] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:13.498350 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e9c53205-a4fd-434c-a0d3-b64696574a4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:13.511419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c53205-a4fd-434c-a0d3-b64696574a4f service placement] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:13.567840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:13.568747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:13.569029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:13.569299 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] PATH_INFO: `/v3/projects/079b28c805454d03b8969138df67321c` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:13.570479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=079b28c805454d03b8969138df67321c)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:13.572447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:13.693481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079b28c805454d03b8969138df67321c', 'request_id': 'req-923b1828-9a0f-4e77-bc6f-2eab466391a1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:13.699553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079b28c805454d03b8969138df67321c', 'request_id': 'req-923b1828-9a0f-4e77-bc6f-2eab466391a1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:13.705044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079b28c805454d03b8969138df67321c', 'request_id': 'req-923b1828-9a0f-4e77-bc6f-2eab466391a1'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:13.772269 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] The token cache is being invalidate because project 079b28c805454d03b8969138df67321c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:04:13.772586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:13.773207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-923b1828-9a0f-4e77-bc6f-2eab466391a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:13.775399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1356/2709] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:04:13 2022] DELETE /identity/v3/projects/079b28c805454d03b8969138df67321c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:13.781290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5ec32-cf53-40a0-a268-36f3b95a9d97 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:13.781780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5ec32-cf53-40a0-a268-36f3b95a9d97 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:13.781888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5ec32-cf53-40a0-a268-36f3b95a9d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:13.799683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-84f5ec32-cf53-40a0-a268-36f3b95a9d97 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:04:13.800900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-84f5ec32-cf53-40a0-a268-36f3b95a9d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:13.857668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1354/2710] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:04:13.921763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218d08e-559f-4f5e-b690-0115e861dfba None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:13.922304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218d08e-559f-4f5e-b690-0115e861dfba None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:13.922304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218d08e-559f-4f5e-b690-0115e861dfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:13.941089 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-4218d08e-559f-4f5e-b690-0115e861dfba None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:04:13.942321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4218d08e-559f-4f5e-b690-0115e861dfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:13.951354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1357/2711] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:04:13.955613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5ec32-cf53-40a0-a268-36f3b95a9d97 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:13.965963 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5ec32-cf53-40a0-a268-36f3b95a9d97 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:13.970599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ccac6-e1be-4dc0-878c-30c19b703610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:13.971322 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ccac6-e1be-4dc0-878c-30c19b703610 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:13.971674 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ccac6-e1be-4dc0-878c-30c19b703610 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:13.972018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ccac6-e1be-4dc0-878c-30c19b703610 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:13.972880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ccac6-e1be-4dc0-878c-30c19b703610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:04:13.973292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ccac6-e1be-4dc0-878c-30c19b703610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:13.975123 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ccac6-e1be-4dc0-878c-30c19b703610 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:13.988560 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1355/2712] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:04:13.992980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218d08e-559f-4f5e-b690-0115e861dfba None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:14.002392 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218d08e-559f-4f5e-b690-0115e861dfba None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:14.007203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671e9e1-b726-42b3-813c-b8289c46346e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:14.007858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671e9e1-b726-42b3-813c-b8289c46346e admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:14.008146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671e9e1-b726-42b3-813c-b8289c46346e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:14.008458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671e9e1-b726-42b3-813c-b8289c46346e admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:14.009701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671e9e1-b726-42b3-813c-b8289c46346e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:14.011616 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671e9e1-b726-42b3-813c-b8289c46346e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:14.129315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1358/2713] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:04:13 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:04:14.133499 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ccac6-e1be-4dc0-878c-30c19b703610 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:14.144615 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ccac6-e1be-4dc0-878c-30c19b703610 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:14.197838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:14.203786 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a25a7-83bf-4661-a484-90a34f01dd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:14.204838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a25a7-83bf-4661-a484-90a34f01dd70 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:14.204838 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a25a7-83bf-4661-a484-90a34f01dd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:14.205082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a25a7-83bf-4661-a484-90a34f01dd70 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:14.205746 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a25a7-83bf-4661-a484-90a34f01dd70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:14.207699 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a25a7-83bf-4661-a484-90a34f01dd70 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:14.325886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1356/2714] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:04:14 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:04:14.330219 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671e9e1-b726-42b3-813c-b8289c46346e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:14.340538 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671e9e1-b726-42b3-813c-b8289c46346e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:14.345131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df02e64-9cab-4330-b34a-c3103254679b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:14.345797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df02e64-9cab-4330-b34a-c3103254679b admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:14.346129 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df02e64-9cab-4330-b34a-c3103254679b admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:14.346399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df02e64-9cab-4330-b34a-c3103254679b admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:14.347152 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df02e64-9cab-4330-b34a-c3103254679b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:14.348802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df02e64-9cab-4330-b34a-c3103254679b admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:14.359357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1359/2715] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:04:14.363634 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a25a7-83bf-4661-a484-90a34f01dd70 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:14.374163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a25a7-83bf-4661-a484-90a34f01dd70 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:14.378782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49863445-d9f9-4663-9a7d-112a83bfd795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:14.379506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49863445-d9f9-4663-9a7d-112a83bfd795 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:14.379738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49863445-d9f9-4663-9a7d-112a83bfd795 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:14.380020 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49863445-d9f9-4663-9a7d-112a83bfd795 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:14.380789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49863445-d9f9-4663-9a7d-112a83bfd795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:14.382376 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49863445-d9f9-4663-9a7d-112a83bfd795 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:14.393591 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1357/2716] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:04:14.397714 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df02e64-9cab-4330-b34a-c3103254679b admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:14.408001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df02e64-9cab-4330-b34a-c3103254679b admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:14.412650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b14dc-3954-4c92-a965-98caf6bac0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:14.413529 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b14dc-3954-4c92-a965-98caf6bac0c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:14.413812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b14dc-3954-4c92-a965-98caf6bac0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:14.414090 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b14dc-3954-4c92-a965-98caf6bac0c2 admin admin] PATH_INFO: `/v3/projects/7edacd3e4f19493fa8d579fb41e419d9/users/90635c3bb7d048b398a0a29b9fc54f6f/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:14.429021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b14dc-3954-4c92-a965-98caf6bac0c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7edacd3e4f19493fa8d579fb41e419d9, user_id=90635c3bb7d048b398a0a29b9fc54f6f, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:14.432334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b14dc-3954-4c92-a965-98caf6bac0c2 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:14.522933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1360/2717] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:04:14 2022] PUT /identity/v3/projects/7edacd3e4f19493fa8d579fb41e419d9/users/90635c3bb7d048b398a0a29b9fc54f6f/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:14.534791 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6989-2e79-4bdb-bbc2-3b1cbbd97fbc None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:14.535085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6989-2e79-4bdb-bbc2-3b1cbbd97fbc None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:14.535364 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6989-2e79-4bdb-bbc2-3b1cbbd97fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:14.563758 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-054f6989-2e79-4bdb-bbc2-3b1cbbd97fbc None None] MFA Rules not processed for user `90635c3bb7d048b398a0a29b9fc54f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:04:14.565049 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-054f6989-2e79-4bdb-bbc2-3b1cbbd97fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:14.629772 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1358/2718] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:04:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:04:14.634133 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b14dc-3954-4c92-a965-98caf6bac0c2 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:14.644482 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:14.696841 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b14dc-3954-4c92-a965-98caf6bac0c2 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:14.702030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31edf4-6a50-40a5-bafe-3c816eea2356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:14.702741 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31edf4-6a50-40a5-bafe-3c816eea2356 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:14.703025 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31edf4-6a50-40a5-bafe-3c816eea2356 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:14.703303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31edf4-6a50-40a5-bafe-3c816eea2356 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:14.704003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31edf4-6a50-40a5-bafe-3c816eea2356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:14.705990 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31edf4-6a50-40a5-bafe-3c816eea2356 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:14.786228 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1361/2719] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:04:14 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:04:14.787720 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f6989-2e79-4bdb-bbc2-3b1cbbd97fbc None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:14.798895 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f6989-2e79-4bdb-bbc2-3b1cbbd97fbc None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:14.855477 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a201-a01e-4d2b-aa66-2831f61b4719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:14.856180 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a201-a01e-4d2b-aa66-2831f61b4719 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:14.856500 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a201-a01e-4d2b-aa66-2831f61b4719 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:14.861605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5a201-a01e-4d2b-aa66-2831f61b4719 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:14.861605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5a201-a01e-4d2b-aa66-2831f61b4719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:14.861605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5a201-a01e-4d2b-aa66-2831f61b4719 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:14.947431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1359/2720] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:04:14 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:04:14.951336 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31edf4-6a50-40a5-bafe-3c816eea2356 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:14.961665 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31edf4-6a50-40a5-bafe-3c816eea2356 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:14.966844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56230a98-ee8b-49ed-a7af-b708f2b96350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:14.967499 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56230a98-ee8b-49ed-a7af-b708f2b96350 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:14.967780 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56230a98-ee8b-49ed-a7af-b708f2b96350 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:14.968056 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56230a98-ee8b-49ed-a7af-b708f2b96350 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:14.968838 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56230a98-ee8b-49ed-a7af-b708f2b96350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:14.970440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56230a98-ee8b-49ed-a7af-b708f2b96350 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:14.982724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1362/2721] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:04:14.986695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a201-a01e-4d2b-aa66-2831f61b4719 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:14.997443 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5a201-a01e-4d2b-aa66-2831f61b4719 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:15.002417 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f3196-6159-4c79-8678-1f545119acf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:15.003134 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f3196-6159-4c79-8678-1f545119acf0 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:15.003415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f3196-6159-4c79-8678-1f545119acf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:15.003694 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f3196-6159-4c79-8678-1f545119acf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:15.004459 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f3196-6159-4c79-8678-1f545119acf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:15.006036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f3196-6159-4c79-8678-1f545119acf0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:15.016189 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1360/2722] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:04:15.020379 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56230a98-ee8b-49ed-a7af-b708f2b96350 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:15.030937 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56230a98-ee8b-49ed-a7af-b708f2b96350 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:15.035486 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc099571-f174-43a6-ba1a-a3ad0bf593a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:15.036376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc099571-f174-43a6-ba1a-a3ad0bf593a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:15.036671 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc099571-f174-43a6-ba1a-a3ad0bf593a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:15.036948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc099571-f174-43a6-ba1a-a3ad0bf593a1 admin admin] PATH_INFO: `/v3/projects/2a7aef9b9a164d0492a3e25f91e3d313/users/33c9b8e755f94ee580bd5bb1a3a71929/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:15.180633 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc099571-f174-43a6-ba1a-a3ad0bf593a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7aef9b9a164d0492a3e25f91e3d313, user_id=33c9b8e755f94ee580bd5bb1a3a71929, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:15.183277 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc099571-f174-43a6-ba1a-a3ad0bf593a1 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:15.249569 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1363/2723] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:04:15 2022] PUT /identity/v3/projects/2a7aef9b9a164d0492a3e25f91e3d313/users/33c9b8e755f94ee580bd5bb1a3a71929/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:15.260735 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731f340-f539-4907-8d99-a4436c3b04e0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:15.261027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731f340-f539-4907-8d99-a4436c3b04e0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:15.261305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731f340-f539-4907-8d99-a4436c3b04e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:15.288011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9731f340-f539-4907-8d99-a4436c3b04e0 None None] MFA Rules not processed for user `33c9b8e755f94ee580bd5bb1a3a71929`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:04:15.289248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9731f340-f539-4907-8d99-a4436c3b04e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:15.322602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:15.354676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1361/2724] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:04:15.359376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc099571-f174-43a6-ba1a-a3ad0bf593a1 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:15.421163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc099571-f174-43a6-ba1a-a3ad0bf593a1 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:15.425670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd8120-1a8a-485f-bc6b-fc88e94a3560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:15.426321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd8120-1a8a-485f-bc6b-fc88e94a3560 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:15.426601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd8120-1a8a-485f-bc6b-fc88e94a3560 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:15.426873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd8120-1a8a-485f-bc6b-fc88e94a3560 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:15.427561 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd8120-1a8a-485f-bc6b-fc88e94a3560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:15.429495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd8120-1a8a-485f-bc6b-fc88e94a3560 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:15.519600 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1364/2725] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:04:15 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:04:15.523564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731f340-f539-4907-8d99-a4436c3b04e0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:15.535061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731f340-f539-4907-8d99-a4436c3b04e0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:15.589446 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742fa5d-dc4f-4bd2-984e-39a09023ecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:15.590113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742fa5d-dc4f-4bd2-984e-39a09023ecce admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:15.590395 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742fa5d-dc4f-4bd2-984e-39a09023ecce admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:15.590671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742fa5d-dc4f-4bd2-984e-39a09023ecce admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:15.591361 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742fa5d-dc4f-4bd2-984e-39a09023ecce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:15.593306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742fa5d-dc4f-4bd2-984e-39a09023ecce admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:15.661194 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1362/2726] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:04:15 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:04:15.665155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd8120-1a8a-485f-bc6b-fc88e94a3560 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:15.675309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd8120-1a8a-485f-bc6b-fc88e94a3560 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:15.679797 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66262c-75ae-4b44-9f53-751bb3f0b8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:15.680476 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66262c-75ae-4b44-9f53-751bb3f0b8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:15.681296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66262c-75ae-4b44-9f53-751bb3f0b8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:15.681572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66262c-75ae-4b44-9f53-751bb3f0b8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:15.682315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66262c-75ae-4b44-9f53-751bb3f0b8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:15.683884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66262c-75ae-4b44-9f53-751bb3f0b8f7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:15.694376 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1365/2727] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:04:15.698958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742fa5d-dc4f-4bd2-984e-39a09023ecce admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:15.709290 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742fa5d-dc4f-4bd2-984e-39a09023ecce admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:15.713884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5003fb-0ebc-4c27-bfc9-46d4f1dfbc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:15.714816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5003fb-0ebc-4c27-bfc9-46d4f1dfbc5e admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:15.715073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5003fb-0ebc-4c27-bfc9-46d4f1dfbc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:15.715353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5003fb-0ebc-4c27-bfc9-46d4f1dfbc5e admin admin] PATH_INFO: `/v3/projects/5b6efbd8d57a4860b01d840fd952db98/users/9464e2ea03764f52a554b9a9e76972de/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:15.730003 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5003fb-0ebc-4c27-bfc9-46d4f1dfbc5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6efbd8d57a4860b01d840fd952db98, user_id=9464e2ea03764f52a554b9a9e76972de, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:15.732667 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5003fb-0ebc-4c27-bfc9-46d4f1dfbc5e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:15.838245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1363/2728] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:04:15 2022] PUT /identity/v3/projects/5b6efbd8d57a4860b01d840fd952db98/users/9464e2ea03764f52a554b9a9e76972de/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:04:15.842145 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66262c-75ae-4b44-9f53-751bb3f0b8f7 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:15.852119 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66262c-75ae-4b44-9f53-751bb3f0b8f7 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:15.882534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:15.909196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd6dee-8ce5-43fc-a196-22cd44608732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:15.909864 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd6dee-8ce5-43fc-a196-22cd44608732 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:15.910178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd6dee-8ce5-43fc-a196-22cd44608732 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:15.910489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd6dee-8ce5-43fc-a196-22cd44608732 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:15.911328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd6dee-8ce5-43fc-a196-22cd44608732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:15.913067 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd6dee-8ce5-43fc-a196-22cd44608732 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:15.923409 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1366/2729] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:04:15.927863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5003fb-0ebc-4c27-bfc9-46d4f1dfbc5e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:15.938739 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5003fb-0ebc-4c27-bfc9-46d4f1dfbc5e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:15.943245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e594ba-b555-4963-9ec1-b1b152a30881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:15.944126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e594ba-b555-4963-9ec1-b1b152a30881 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:15.944439 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e594ba-b555-4963-9ec1-b1b152a30881 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:15.944716 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e594ba-b555-4963-9ec1-b1b152a30881 admin admin] PATH_INFO: `/v3/projects/5b6efbd8d57a4860b01d840fd952db98/users/9464e2ea03764f52a554b9a9e76972de/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:15.947239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e594ba-b555-4963-9ec1-b1b152a30881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6efbd8d57a4860b01d840fd952db98, user_id=9464e2ea03764f52a554b9a9e76972de, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:15.949941 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e594ba-b555-4963-9ec1-b1b152a30881 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:15.960141 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1364/2730] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:04:15 2022] PUT /identity/v3/projects/5b6efbd8d57a4860b01d840fd952db98/users/9464e2ea03764f52a554b9a9e76972de/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:15.971514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa9c4c-ac97-4c70-a23a-1623d5c7b073 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:15.972005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa9c4c-ac97-4c70-a23a-1623d5c7b073 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:15.972117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa9c4c-ac97-4c70-a23a-1623d5c7b073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:15.999164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-baaa9c4c-ac97-4c70-a23a-1623d5c7b073 None None] MFA Rules not processed for user `9464e2ea03764f52a554b9a9e76972de`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:04:16.000347 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-baaa9c4c-ac97-4c70-a23a-1623d5c7b073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:16.066058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1367/2731] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:04:16.077409 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e594ba-b555-4963-9ec1-b1b152a30881 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:16.079099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-30e594ba-b555-4963-9ec1-b1b152a30881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:16.135187 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e594ba-b555-4963-9ec1-b1b152a30881 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:16.138749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee630d-490d-4820-a90c-d3922567fdc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:16.139386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee630d-490d-4820-a90c-d3922567fdc5 service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:16.139670 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee630d-490d-4820-a90c-d3922567fdc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:16.139946 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee630d-490d-4820-a90c-d3922567fdc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:16.179679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:16.197287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee630d-490d-4820-a90c-d3922567fdc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:16.198725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee630d-490d-4820-a90c-d3922567fdc5 service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:16.207943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1365/2732] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:04:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:04:18.172629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa9c4c-ac97-4c70-a23a-1623d5c7b073 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:18.174042 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-baaa9c4c-ac97-4c70-a23a-1623d5c7b073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:18.186320 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa9c4c-ac97-4c70-a23a-1623d5c7b073 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:18.190018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244632a-9981-4dc9-a247-37ba6299f3d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:18.190660 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244632a-9981-4dc9-a247-37ba6299f3d6 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:18.190941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244632a-9981-4dc9-a247-37ba6299f3d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:18.191212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244632a-9981-4dc9-a247-37ba6299f3d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:18.247914 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244632a-9981-4dc9-a247-37ba6299f3d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:18.249389 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244632a-9981-4dc9-a247-37ba6299f3d6 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:18.258017 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1368/2733] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:04:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:04:19.581719 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee630d-490d-4820-a90c-d3922567fdc5 service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:19.590423 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee630d-490d-4820-a90c-d3922567fdc5 service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:19.594027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3bbc0-4c90-4675-9fa8-3ac44b3b8d5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:19.594664 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3bbc0-4c90-4675-9fa8-3ac44b3b8d5e service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:19.594945 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3bbc0-4c90-4675-9fa8-3ac44b3b8d5e service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:19.595221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3bbc0-4c90-4675-9fa8-3ac44b3b8d5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:19.604031 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3bbc0-4c90-4675-9fa8-3ac44b3b8d5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:19.605517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3bbc0-4c90-4675-9fa8-3ac44b3b8d5e service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:19.616140 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1366/2734] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:04:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:04:24.334921 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244632a-9981-4dc9-a247-37ba6299f3d6 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:24.385801 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:24.396481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244632a-9981-4dc9-a247-37ba6299f3d6 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:24.401676 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:24.402537 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:24.402846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:24.403123 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] PATH_INFO: `/v3/users/90635c3bb7d048b398a0a29b9fc54f6f` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:24.404298 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90635c3bb7d048b398a0a29b9fc54f6f)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:24.406271 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:24.582966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90635c3bb7d048b398a0a29b9fc54f6f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:24.583540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90635c3bb7d048b398a0a29b9fc54f6f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:24.584132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90635c3bb7d048b398a0a29b9fc54f6f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:24.591165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90635c3bb7d048b398a0a29b9fc54f6f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:24.603257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90635c3bb7d048b398a0a29b9fc54f6f'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:24.698956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1369/2735] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:04:24 2022] DELETE /identity/v3/users/90635c3bb7d048b398a0a29b9fc54f6f => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:24.703762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02497b-a553-42b6-b62b-1cfc438070e2 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:24.706841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02497b-a553-42b6-b62b-1cfc438070e2 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:24.706841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02497b-a553-42b6-b62b-1cfc438070e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:24.721525 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9a02497b-a553-42b6-b62b-1cfc438070e2 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:04:24.722787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9a02497b-a553-42b6-b62b-1cfc438070e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:24.779346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1367/2736] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:04:24.788910 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:24.790305 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:24.846928 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ece17-1c84-4045-9246-e13c25b16851 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:24.850412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3b882-9ddb-4293-9af3-9ad9ccf7d34f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:24.851050 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3b882-9ddb-4293-9af3-9ad9ccf7d34f service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:24.851330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3b882-9ddb-4293-9af3-9ad9ccf7d34f service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:24.851597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3b882-9ddb-4293-9af3-9ad9ccf7d34f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:24.860226 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3b882-9ddb-4293-9af3-9ad9ccf7d34f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:24.861671 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3b882-9ddb-4293-9af3-9ad9ccf7d34f service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:24.871666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1370/2737] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:04:25.283550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02497b-a553-42b6-b62b-1cfc438070e2 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:25.285021 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9a02497b-a553-42b6-b62b-1cfc438070e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:25.298950 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02497b-a553-42b6-b62b-1cfc438070e2 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:25.303846 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:25.304747 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:25.305033 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:25.305314 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] PATH_INFO: `/v3/projects/7edacd3e4f19493fa8d579fb41e419d9` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:25.307158 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7edacd3e4f19493fa8d579fb41e419d9)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:25.309107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:25.414009 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7edacd3e4f19493fa8d579fb41e419d9', 'request_id': 'req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:25.420904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7edacd3e4f19493fa8d579fb41e419d9', 'request_id': 'req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:25.426445 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7edacd3e4f19493fa8d579fb41e419d9', 'request_id': 'req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:25.503696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] The token cache is being invalidate because project 7edacd3e4f19493fa8d579fb41e419d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:04:25.503985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:25.504627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:25.506191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1368/2738] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:04:25 2022] DELETE /identity/v3/projects/7edacd3e4f19493fa8d579fb41e419d9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:25.510290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3b882-9ddb-4293-9af3-9ad9ccf7d34f service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:25.519775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3b882-9ddb-4293-9af3-9ad9ccf7d34f service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:25.574032 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:25.574885 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:25.575166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:25.575440 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] PATH_INFO: `/v3/users/33c9b8e755f94ee580bd5bb1a3a71929` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:25.576634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c9b8e755f94ee580bd5bb1a3a71929)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:25.578535 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:25.593898 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:25.774304 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c9b8e755f94ee580bd5bb1a3a71929'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:25.775582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c9b8e755f94ee580bd5bb1a3a71929'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:25.776193 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c9b8e755f94ee580bd5bb1a3a71929'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:25.783005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c9b8e755f94ee580bd5bb1a3a71929'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:25.795153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c9b8e755f94ee580bd5bb1a3a71929'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:25.904732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1371/2739] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:04:25 2022] DELETE /identity/v3/users/33c9b8e755f94ee580bd5bb1a3a71929 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:26.305725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:26.307734 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:26.322508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4066a0-ff36-4381-8a49-b3d3b2acffa2 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:26.370113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:26.379479 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:26.380426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:26.380715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:26.380997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] PATH_INFO: `/v3/projects/2a7aef9b9a164d0492a3e25f91e3d313` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:26.382174 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7aef9b9a164d0492a3e25f91e3d313)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:26.384190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:26.471886 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7aef9b9a164d0492a3e25f91e3d313', 'request_id': 'req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:26.478546 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7aef9b9a164d0492a3e25f91e3d313', 'request_id': 'req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:26.483962 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7aef9b9a164d0492a3e25f91e3d313', 'request_id': 'req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:26.563205 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] The token cache is being invalidate because project 2a7aef9b9a164d0492a3e25f91e3d313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:04:26.563433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:26.564052 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:26.565777 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1369/2740] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:04:26 2022] DELETE /identity/v3/projects/2a7aef9b9a164d0492a3e25f91e3d313 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:26.572378 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:26.573817 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:26.586342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538d8e3-75af-4c8d-92f6-72550e4db9a5 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:26.641319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:26.642173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:26.642451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:26.642724 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] PATH_INFO: `/v3/users/9464e2ea03764f52a554b9a9e76972de` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:26.643887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9464e2ea03764f52a554b9a9e76972de)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:26.645808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:26.782270 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9464e2ea03764f52a554b9a9e76972de'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:26.782926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9464e2ea03764f52a554b9a9e76972de'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:26.783536 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9464e2ea03764f52a554b9a9e76972de'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:26.790495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9464e2ea03764f52a554b9a9e76972de'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:26.802669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-49797ec0-8325-4d34-ab15-14d103830389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9464e2ea03764f52a554b9a9e76972de'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:26.895012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1372/2741] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:04:26 2022] DELETE /identity/v3/users/9464e2ea03764f52a554b9a9e76972de => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:27.307465 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:27.309061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:27.323541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82f21e-a61b-47ef-bed7-1ba7cb2c4265 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:27.379712 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:27.380606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:27.380898 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:27.381199 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] PATH_INFO: `/v3/projects/5b6efbd8d57a4860b01d840fd952db98` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:27.382367 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b6efbd8d57a4860b01d840fd952db98)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:27.384259 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:27.476197 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b6efbd8d57a4860b01d840fd952db98', 'request_id': 'req-08c83138-0ad5-49ea-863d-89e719fed2f2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:27.482190 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b6efbd8d57a4860b01d840fd952db98', 'request_id': 'req-08c83138-0ad5-49ea-863d-89e719fed2f2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:27.488063 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b6efbd8d57a4860b01d840fd952db98', 'request_id': 'req-08c83138-0ad5-49ea-863d-89e719fed2f2'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:27.572790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] The token cache is being invalidate because project 5b6efbd8d57a4860b01d840fd952db98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:04:27.573077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:27.573702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:04:27.575261 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1370/2742] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:04:27 2022] DELETE /identity/v3/projects/5b6efbd8d57a4860b01d840fd952db98 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:04:36.891065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:DELETE=19,keystone:SELECT=73,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:37.568978 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=79,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:45.512500 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7947992-8dda-4a63-96b1-3e03bb1f4a92 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:45.513022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7947992-8dda-4a63-96b1-3e03bb1f4a92 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:45.513136 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7947992-8dda-4a63-96b1-3e03bb1f4a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:45.530889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-a7947992-8dda-4a63-96b1-3e03bb1f4a92 None None] MFA Rules not processed for user `929c01c856004eb68f6eb397e9976309`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:04:45.532104 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a7947992-8dda-4a63-96b1-3e03bb1f4a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:45.587280 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1373/2743] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:04:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:04:46.150748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:04:46.152224 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:46.210119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c83138-0ad5-49ea-863d-89e719fed2f2 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:04:46.213851 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f249e-5a3a-489c-9d0a-6ad1c8b433db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:04:46.214506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f249e-5a3a-489c-9d0a-6ad1c8b433db service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:04:46.214884 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f249e-5a3a-489c-9d0a-6ad1c8b433db service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:04:46.215166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f249e-5a3a-489c-9d0a-6ad1c8b433db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:04:46.217028 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d72f249e-5a3a-489c-9d0a-6ad1c8b433db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:04:46.226765 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f249e-5a3a-489c-9d0a-6ad1c8b433db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:04:46.228172 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f249e-5a3a-489c-9d0a-6ad1c8b433db service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:04:46.236900 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1371/2744] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:04:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:04:55.580749 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=25 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:04:56.232660 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=26 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:07.280097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7947992-8dda-4a63-96b1-3e03bb1f4a92 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:05:07.281543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-a7947992-8dda-4a63-96b1-3e03bb1f4a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:05:07.337518 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7947992-8dda-4a63-96b1-3e03bb1f4a92 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:05:07.341109 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b693a0-08c5-4b7f-8464-a87507d0e980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:05:07.341761 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b693a0-08c5-4b7f-8464-a87507d0e980 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:05:07.342047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b693a0-08c5-4b7f-8464-a87507d0e980 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:05:07.342322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b693a0-08c5-4b7f-8464-a87507d0e980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:05:07.344218 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-25b693a0-08c5-4b7f-8464-a87507d0e980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:05:07.402512 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b693a0-08c5-4b7f-8464-a87507d0e980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:05:07.403936 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b693a0-08c5-4b7f-8464-a87507d0e980 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:05:07.413165 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1374/2745] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:05:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:05:15.795186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3040395-cf33-4647-8388-25d5e3fe4551 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:05:15.797240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3040395-cf33-4647-8388-25d5e3fe4551 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:05:15.797240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3040395-cf33-4647-8388-25d5e3fe4551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:05:15.814582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-b3040395-cf33-4647-8388-25d5e3fe4551 None None] MFA Rules not processed for user `7e03ea067e27456590c0865766830d73`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:05:15.815827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b3040395-cf33-4647-8388-25d5e3fe4551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:05:15.823625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1372/2746] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:05:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:05:17.408707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=48 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:25.807793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:05:58.108889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fad16-6931-4d8e-a7ad-892d28a471a8 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:05:58.110274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fad16-6931-4d8e-a7ad-892d28a471a8 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:05:58.110568 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fad16-6931-4d8e-a7ad-892d28a471a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:05:58.128690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-d05fad16-6931-4d8e-a7ad-892d28a471a8 None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:05:58.129946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-d05fad16-6931-4d8e-a7ad-892d28a471a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:05:58.183964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1375/2747] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:05:58.335431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3040395-cf33-4647-8388-25d5e3fe4551 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:05:58.336891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b3040395-cf33-4647-8388-25d5e3fe4551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:05:58.408144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3040395-cf33-4647-8388-25d5e3fe4551 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:05:58.413342 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881d9c1-47eb-4713-9ac7-5f3cce8b4fc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:05:58.413997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881d9c1-47eb-4713-9ac7-5f3cce8b4fc5 service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:05:58.414287 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881d9c1-47eb-4713-9ac7-5f3cce8b4fc5 service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:05:58.414571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881d9c1-47eb-4713-9ac7-5f3cce8b4fc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:05:58.423775 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881d9c1-47eb-4713-9ac7-5f3cce8b4fc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:05:58.425425 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881d9c1-47eb-4713-9ac7-5f3cce8b4fc5 service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:05:58.436064 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1373/2748] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 16:05:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 16:06:05.685729 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fad16-6931-4d8e-a7ad-892d28a471a8 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:06:05.697625 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fad16-6931-4d8e-a7ad-892d28a471a8 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:06:05.701944 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71a1eb-0c9c-4001-ad8d-69c275027234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:06:05.702623 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71a1eb-0c9c-4001-ad8d-69c275027234 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:05.702907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71a1eb-0c9c-4001-ad8d-69c275027234 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:05.703245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71a1eb-0c9c-4001-ad8d-69c275027234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:05.705112 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3b71a1eb-0c9c-4001-ad8d-69c275027234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:06:05.791235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71a1eb-0c9c-4001-ad8d-69c275027234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:06:05.792735 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71a1eb-0c9c-4001-ad8d-69c275027234 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:06:05.808746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1376/2749] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:06:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:06:08.432170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=32 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:09.317345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881d9c1-47eb-4713-9ac7-5f3cce8b4fc5 service swift] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:06:09.327506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881d9c1-47eb-4713-9ac7-5f3cce8b4fc5 service swift] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:06:09.331374 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a54d0-724e-43e5-8036-424c5de076e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:06:09.332027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a54d0-724e-43e5-8036-424c5de076e8 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:09.332310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a54d0-724e-43e5-8036-424c5de076e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:09.332619 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a54d0-724e-43e5-8036-424c5de076e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:09.334442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-955a54d0-724e-43e5-8036-424c5de076e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:06:09.347938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a54d0-724e-43e5-8036-424c5de076e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:06:09.349386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a54d0-724e-43e5-8036-424c5de076e8 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:06:09.360715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1374/2750] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:06:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:06:13.929118 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7947f3-03d6-4a98-9244-d43abb285472 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:13.929659 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7947f3-03d6-4a98-9244-d43abb285472 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:13.930006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7947f3-03d6-4a98-9244-d43abb285472 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:13.932120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1377/2751] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 16:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:06:13.938307 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d26bb-1333-4d82-9443-f210a1b73cde None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:13.938793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d26bb-1333-4d82-9443-f210a1b73cde None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:13.939017 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d26bb-1333-4d82-9443-f210a1b73cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:13.957429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-a22d26bb-1333-4d82-9443-f210a1b73cde None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:06:13.958683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a22d26bb-1333-4d82-9443-f210a1b73cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:06:13.968107 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1375/2752] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 16:06:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:06:13.980261 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7947f3-03d6-4a98-9244-d43abb285472 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:06:13.982572 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-cb7947f3-03d6-4a98-9244-d43abb285472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:06:14.040046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7947f3-03d6-4a98-9244-d43abb285472 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:06:14.043575 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2d54d-96be-48cd-89a6-554b0c0fcef9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '142e6e44d6544f6a891ed3f913de74ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:06:14.044214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2d54d-96be-48cd-89a6-554b0c0fcef9 service glance] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:14.045065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2d54d-96be-48cd-89a6-554b0c0fcef9 service glance] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:14.045342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2d54d-96be-48cd-89a6-554b0c0fcef9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:14.055197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2d54d-96be-48cd-89a6-554b0c0fcef9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:06:14.056645 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2d54d-96be-48cd-89a6-554b0c0fcef9 service glance] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:06:14.067698 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1378/2753] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:06:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:06:14.247130 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d5ffd-197e-4f0e-a9b8-efef03ca5432 None None] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:14.247458 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d5ffd-197e-4f0e-a9b8-efef03ca5432 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:14.248304 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d5ffd-197e-4f0e-a9b8-efef03ca5432 None None] PATH_INFO: `/` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:14.249715 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1376/2754] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 16:06:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:06:14.254497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148215e4-8d51-43a2-a18f-7cb396cb2d56 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:14.254814 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148215e4-8d51-43a2-a18f-7cb396cb2d56 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:14.255097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148215e4-8d51-43a2-a18f-7cb396cb2d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:14.291172 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-148215e4-8d51-43a2-a18f-7cb396cb2d56 None None] MFA Rules not processed for user `a39f5189d63d467dbd914f553c261c01`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:06:14.292391 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-148215e4-8d51-43a2-a18f-7cb396cb2d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:06:14.299287 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1379/2755] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 16:06:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:06:14.311440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d5ffd-197e-4f0e-a9b8-efef03ca5432 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:06:14.322401 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d5ffd-197e-4f0e-a9b8-efef03ca5432 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:06:14.325881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d2b88-8bda-41da-90e0-598c9dea7a5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:06:14.326506 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2b88-8bda-41da-90e0-598c9dea7a5c service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:14.326793 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2b88-8bda-41da-90e0-598c9dea7a5c service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:14.327067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d2b88-8bda-41da-90e0-598c9dea7a5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:14.333938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d2b88-8bda-41da-90e0-598c9dea7a5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:06:14.335334 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d2b88-8bda-41da-90e0-598c9dea7a5c service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:06:14.343558 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1377/2756] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:06:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:06:14.425510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031c482-3ed6-40e7-b42a-931262371933 None None] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:14.425807 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031c482-3ed6-40e7-b42a-931262371933 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:14.426096 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031c482-3ed6-40e7-b42a-931262371933 None None] PATH_INFO: `/` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:14.427553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1380/2757] 10.222.0.23 () {54 vars in 921 bytes} [Mon May 23 16:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:06:14.432915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319b782-7194-46c9-981b-0cc35d89f693 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:14.435709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319b782-7194-46c9-981b-0cc35d89f693 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:14.435709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319b782-7194-46c9-981b-0cc35d89f693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:14.451832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-9319b782-7194-46c9-981b-0cc35d89f693 None None] MFA Rules not processed for user `b9129f0b362a4945a0eca7cc33571abd`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:06:14.453175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-9319b782-7194-46c9-981b-0cc35d89f693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:06:14.462529 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1378/2758] 10.222.0.23 () {58 vars in 1018 bytes} [Mon May 23 16:06:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:06:14.473467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031c482-3ed6-40e7-b42a-931262371933 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:06:14.485549 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031c482-3ed6-40e7-b42a-931262371933 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:06:14.489174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c85b37-0f8d-42d2-b326-b2a037b08b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:06:14.489800 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c85b37-0f8d-42d2-b326-b2a037b08b4b service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:14.490067 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c85b37-0f8d-42d2-b326-b2a037b08b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:14.490342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c85b37-0f8d-42d2-b326-b2a037b08b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:14.500563 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c85b37-0f8d-42d2-b326-b2a037b08b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:06:14.501980 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c85b37-0f8d-42d2-b326-b2a037b08b4b service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:06:14.508120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:14.513580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1381/2759] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:06:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:06:16.671996 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b54c4-3cff-4de7-a5cb-4113b1afc1e9 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:16.672488 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b54c4-3cff-4de7-a5cb-4113b1afc1e9 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:16.672593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b54c4-3cff-4de7-a5cb-4113b1afc1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:16.690385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-915b54c4-3cff-4de7-a5cb-4113b1afc1e9 None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:06:16.691555 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-915b54c4-3cff-4de7-a5cb-4113b1afc1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:06:16.698508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1379/2760] 10.222.0.23 () {58 vars in 1009 bytes} [Mon May 23 16:06:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:06:16.710717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c85b37-0f8d-42d2-b326-b2a037b08b4b service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:06:16.721891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c85b37-0f8d-42d2-b326-b2a037b08b4b service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:06:16.726789 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34c7c4-a164-436f-bc8a-214b2d403e2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:06:16.727420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34c7c4-a164-436f-bc8a-214b2d403e2d service swift] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:16.727707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34c7c4-a164-436f-bc8a-214b2d403e2d service swift] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:16.727986 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34c7c4-a164-436f-bc8a-214b2d403e2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:16.736312 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34c7c4-a164-436f-bc8a-214b2d403e2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:06:16.737925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34c7c4-a164-436f-bc8a-214b2d403e2d service swift] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:06:16.746008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1382/2761] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 16:06:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 16:06:26.684711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=21 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:26.742743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=6 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:29.802835 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b54c4-3cff-4de7-a5cb-4113b1afc1e9 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:06:29.812015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b54c4-3cff-4de7-a5cb-4113b1afc1e9 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:06:29.815496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4704f-5c9b-4d06-8b98-0cc6ec23f658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:06:29.816129 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4704f-5c9b-4d06-8b98-0cc6ec23f658 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:29.816442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4704f-5c9b-4d06-8b98-0cc6ec23f658 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:29.816722 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4704f-5c9b-4d06-8b98-0cc6ec23f658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:29.818626 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6bc4704f-5c9b-4d06-8b98-0cc6ec23f658 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:06:29.828267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4704f-5c9b-4d06-8b98-0cc6ec23f658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:06:29.830266 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4704f-5c9b-4d06-8b98-0cc6ec23f658 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:06:29.838394 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1380/2762] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:06:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:06:31.407235 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b84fa-840a-4ad6-81a6-bbda7934014b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:31.407719 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b84fa-840a-4ad6-81a6-bbda7934014b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:31.407840 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b84fa-840a-4ad6-81a6-bbda7934014b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:31.425725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e22b84fa-840a-4ad6-81a6-bbda7934014b None None] MFA Rules not processed for user `72e29fe3a2b645248eef1d52217324d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:06:31.427143 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e22b84fa-840a-4ad6-81a6-bbda7934014b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:06:31.434021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1383/2763] 10.222.0.23 () {58 vars in 967 bytes} [Mon May 23 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:06:31.446629 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4704f-5c9b-4d06-8b98-0cc6ec23f658 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:06:31.458627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4704f-5c9b-4d06-8b98-0cc6ec23f658 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:06:31.463559 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990902d-9f4a-46c1-ac69-be7cee141d43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57de80064b74ad59d33abb83a5d17f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:06:31.464198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990902d-9f4a-46c1-ac69-be7cee141d43 service swift] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:06:31.464581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990902d-9f4a-46c1-ac69-be7cee141d43 service swift] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:06:31.464867 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990902d-9f4a-46c1-ac69-be7cee141d43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:06:31.473611 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990902d-9f4a-46c1-ac69-be7cee141d43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:06:31.475204 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990902d-9f4a-46c1-ac69-be7cee141d43 service swift] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:06:31.484788 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1381/2764] 10.222.0.23 () {60 vars in 1384 bytes} [Mon May 23 16:06:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 16:06:41.420470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:06:41.480041 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=12 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:01.325083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b84fa-840a-4ad6-81a6-bbda7934014b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:01.326470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e22b84fa-840a-4ad6-81a6-bbda7934014b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:01.338152 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b84fa-840a-4ad6-81a6-bbda7934014b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:01.341715 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876783e-e3a0-44e8-8af2-63d491a72d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:01.342349 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876783e-e3a0-44e8-8af2-63d491a72d56 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:01.342633 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876783e-e3a0-44e8-8af2-63d491a72d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:01.342909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876783e-e3a0-44e8-8af2-63d491a72d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:01.344683 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6876783e-e3a0-44e8-8af2-63d491a72d56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:01.355711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876783e-e3a0-44e8-8af2-63d491a72d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:01.357157 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876783e-e3a0-44e8-8af2-63d491a72d56 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:01.366200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1384/2765] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:07:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:07:04.303036 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990902d-9f4a-46c1-ac69-be7cee141d43 service swift] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:04.304643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7990902d-9f4a-46c1-ac69-be7cee141d43 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:04.332482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990902d-9f4a-46c1-ac69-be7cee141d43 service swift] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:04.343881 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f15ea6-c03e-4684-b603-9342d8163d6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:04.344804 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f15ea6-c03e-4684-b603-9342d8163d6d service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:04.345318 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f15ea6-c03e-4684-b603-9342d8163d6d service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:04.345671 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f15ea6-c03e-4684-b603-9342d8163d6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:04.348131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c3f15ea6-c03e-4684-b603-9342d8163d6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:04.362768 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f15ea6-c03e-4684-b603-9342d8163d6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:04.364515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f15ea6-c03e-4684-b603-9342d8163d6d service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:04.377862 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1382/2766] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:07:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:07:08.877829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876783e-e3a0-44e8-8af2-63d491a72d56 service cinder] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:08.887146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876783e-e3a0-44e8-8af2-63d491a72d56 service cinder] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:08.891339 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89ce9f-09a9-4ffc-8b2f-c06574d46eb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:08.891967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89ce9f-09a9-4ffc-8b2f-c06574d46eb7 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:08.892248 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89ce9f-09a9-4ffc-8b2f-c06574d46eb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:08.892551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89ce9f-09a9-4ffc-8b2f-c06574d46eb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:08.894288 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-6a89ce9f-09a9-4ffc-8b2f-c06574d46eb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:08.954852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89ce9f-09a9-4ffc-8b2f-c06574d46eb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:08.956300 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89ce9f-09a9-4ffc-8b2f-c06574d46eb7 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:08.965819 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1385/2767] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:07:12.371122 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f15ea6-c03e-4684-b603-9342d8163d6d service nova] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:12.372632 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c3f15ea6-c03e-4684-b603-9342d8163d6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:12.386545 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f15ea6-c03e-4684-b603-9342d8163d6d service nova] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:12.391832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:12.392728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:12.393013 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:12.393294 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] PATH_INFO: `/v3/users/929c01c856004eb68f6eb397e9976309` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:12.394481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=929c01c856004eb68f6eb397e9976309)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:12.396524 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:12.576069 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '929c01c856004eb68f6eb397e9976309'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:12.576755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '929c01c856004eb68f6eb397e9976309'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:12.589343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '929c01c856004eb68f6eb397e9976309'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:12.590118 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '929c01c856004eb68f6eb397e9976309'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:12.597011 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-8de2a70f-1cde-4fbb-8b05-d128f3e32a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '929c01c856004eb68f6eb397e9976309'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:12.683703 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1383/2768] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:07:12 2022] DELETE /identity/v3/users/929c01c856004eb68f6eb397e9976309 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:07:12.931985 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89ce9f-09a9-4ffc-8b2f-c06574d46eb7 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:12.944263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89ce9f-09a9-4ffc-8b2f-c06574d46eb7 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:13.002810 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:13.003705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:13.003982 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:13.004254 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] PATH_INFO: `/v3/projects/fc4de09e5a074e53a8eb787794367e74` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:13.005454 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4de09e5a074e53a8eb787794367e74)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:13.007322 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:13.107184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4de09e5a074e53a8eb787794367e74', 'request_id': 'req-71e127e1-eba4-4c36-8beb-007ba9507b1e'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:13.114404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4de09e5a074e53a8eb787794367e74', 'request_id': 'req-71e127e1-eba4-4c36-8beb-007ba9507b1e'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:13.120059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4de09e5a074e53a8eb787794367e74', 'request_id': 'req-71e127e1-eba4-4c36-8beb-007ba9507b1e'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:13.225249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] The token cache is being invalidate because project fc4de09e5a074e53a8eb787794367e74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:07:13.225565 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:13.233141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-71e127e1-eba4-4c36-8beb-007ba9507b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:13.233141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1386/2769] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:07:12 2022] DELETE /identity/v3/projects/fc4de09e5a074e53a8eb787794367e74 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:07:13.233348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b3470-b42f-40e4-9e88-23b7d7bcb3dd None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:13.233348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b3470-b42f-40e4-9e88-23b7d7bcb3dd None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:13.233616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b3470-b42f-40e4-9e88-23b7d7bcb3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:13.253239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-ce2b3470-b42f-40e4-9e88-23b7d7bcb3dd None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:07:13.254437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ce2b3470-b42f-40e4-9e88-23b7d7bcb3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:13.317166 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1384/2770] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:07:13.324855 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe88eb8-d64f-4fd5-8d21-c2463f4649d6 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:13.325150 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe88eb8-d64f-4fd5-8d21-c2463f4649d6 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:13.325428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe88eb8-d64f-4fd5-8d21-c2463f4649d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:13.343207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-bbe88eb8-d64f-4fd5-8d21-c2463f4649d6 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:07:13.344428 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bbe88eb8-d64f-4fd5-8d21-c2463f4649d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:13.352994 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1387/2771] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:07:13.357007 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b3470-b42f-40e4-9e88-23b7d7bcb3dd None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:13.367198 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b3470-b42f-40e4-9e88-23b7d7bcb3dd None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:13.371755 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a8a64-2aaf-4c2b-9126-23348ef3fa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:13.372433 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8a64-2aaf-4c2b-9126-23348ef3fa3c admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:13.372717 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8a64-2aaf-4c2b-9126-23348ef3fa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:13.372989 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8a64-2aaf-4c2b-9126-23348ef3fa3c admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:13.373709 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a8a64-2aaf-4c2b-9126-23348ef3fa3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:07:13.374015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a8a64-2aaf-4c2b-9126-23348ef3fa3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:13.375593 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a8a64-2aaf-4c2b-9126-23348ef3fa3c admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:13.387042 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1385/2772] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:07:13.391087 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe88eb8-d64f-4fd5-8d21-c2463f4649d6 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:13.396644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:13.400554 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe88eb8-d64f-4fd5-8d21-c2463f4649d6 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:13.405166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710aa0e-bf57-4910-a407-e8a2e0cde7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:13.409859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710aa0e-bf57-4910-a407-e8a2e0cde7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:13.409859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710aa0e-bf57-4910-a407-e8a2e0cde7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:13.409859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710aa0e-bf57-4910-a407-e8a2e0cde7d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:13.409859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710aa0e-bf57-4910-a407-e8a2e0cde7d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:13.409859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710aa0e-bf57-4910-a407-e8a2e0cde7d7 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:13.512873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1388/2773] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:07:13 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:07:13.517622 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a8a64-2aaf-4c2b-9126-23348ef3fa3c admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:13.527726 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a8a64-2aaf-4c2b-9126-23348ef3fa3c admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:13.571265 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:13.584353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fc3ee-2ce1-4978-ab92-9abc825cf627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:13.585059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc3ee-2ce1-4978-ab92-9abc825cf627 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:13.585350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc3ee-2ce1-4978-ab92-9abc825cf627 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:13.585634 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fc3ee-2ce1-4978-ab92-9abc825cf627 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:13.586329 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fc3ee-2ce1-4978-ab92-9abc825cf627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:13.590245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fc3ee-2ce1-4978-ab92-9abc825cf627 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:13.713854 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1386/2774] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:07:13 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:07:13.717930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710aa0e-bf57-4910-a407-e8a2e0cde7d7 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:13.729098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710aa0e-bf57-4910-a407-e8a2e0cde7d7 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:13.735205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eec725-747a-430d-828a-0b9f1d7f8619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:13.735902 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eec725-747a-430d-828a-0b9f1d7f8619 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:13.736195 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eec725-747a-430d-828a-0b9f1d7f8619 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:13.736525 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eec725-747a-430d-828a-0b9f1d7f8619 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:13.737267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eec725-747a-430d-828a-0b9f1d7f8619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:13.738878 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eec725-747a-430d-828a-0b9f1d7f8619 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:13.749913 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1389/2775] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:07:13.754065 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fc3ee-2ce1-4978-ab92-9abc825cf627 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:13.764475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fc3ee-2ce1-4978-ab92-9abc825cf627 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:13.770289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ff802-7742-4177-a255-a2964350e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:13.770864 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ff802-7742-4177-a255-a2964350e69f admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:13.771151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ff802-7742-4177-a255-a2964350e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:13.771429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ff802-7742-4177-a255-a2964350e69f admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:13.772157 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ff802-7742-4177-a255-a2964350e69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:13.773775 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ff802-7742-4177-a255-a2964350e69f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:13.784687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1387/2776] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:07:13.789027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eec725-747a-430d-828a-0b9f1d7f8619 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:13.799080 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eec725-747a-430d-828a-0b9f1d7f8619 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:13.803586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957fb4f-e296-4238-9183-cb21f66c6345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:13.804492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957fb4f-e296-4238-9183-cb21f66c6345 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:13.804782 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957fb4f-e296-4238-9183-cb21f66c6345 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:13.805062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957fb4f-e296-4238-9183-cb21f66c6345 admin admin] PATH_INFO: `/v3/projects/f7e46ae2884d4ae48592abdd6569e499/users/0f25bf41a74d43978435bcb856863b5b/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:13.819414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957fb4f-e296-4238-9183-cb21f66c6345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e46ae2884d4ae48592abdd6569e499, user_id=0f25bf41a74d43978435bcb856863b5b, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:13.822088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957fb4f-e296-4238-9183-cb21f66c6345 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:13.884016 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1390/2777] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:07:13 2022] PUT /identity/v3/projects/f7e46ae2884d4ae48592abdd6569e499/users/0f25bf41a74d43978435bcb856863b5b/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:07:13.889121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67496877-e346-4eb1-a0d3-94f069b5a012 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:13.889416 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67496877-e346-4eb1-a0d3-94f069b5a012 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:13.889697 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67496877-e346-4eb1-a0d3-94f069b5a012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:13.907748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-67496877-e346-4eb1-a0d3-94f069b5a012 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:07:13.908985 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-67496877-e346-4eb1-a0d3-94f069b5a012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:13.966650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1388/2778] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:07:13.976079 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957fb4f-e296-4238-9183-cb21f66c6345 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:13.977823 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2957fb4f-e296-4238-9183-cb21f66c6345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:14.035489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957fb4f-e296-4238-9183-cb21f66c6345 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:14.039197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013115b2-c823-48d7-927a-70a5464faa02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:14.039847 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013115b2-c823-48d7-927a-70a5464faa02 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:14.040134 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013115b2-c823-48d7-927a-70a5464faa02 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:14.040444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013115b2-c823-48d7-927a-70a5464faa02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:14.049478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013115b2-c823-48d7-927a-70a5464faa02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:14.050909 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013115b2-c823-48d7-927a-70a5464faa02 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:14.060296 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1391/2779] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:07:18.486969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67496877-e346-4eb1-a0d3-94f069b5a012 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:18.488463 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-67496877-e346-4eb1-a0d3-94f069b5a012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:18.590381 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67496877-e346-4eb1-a0d3-94f069b5a012 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:18.597481 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace8144-b4e2-4a15-8e86-1a045b7da0ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:18.598317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace8144-b4e2-4a15-8e86-1a045b7da0ed service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:18.598710 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace8144-b4e2-4a15-8e86-1a045b7da0ed service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:18.599119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace8144-b4e2-4a15-8e86-1a045b7da0ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:18.601245 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-4ace8144-b4e2-4a15-8e86-1a045b7da0ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:18.615724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace8144-b4e2-4a15-8e86-1a045b7da0ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:18.617764 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace8144-b4e2-4a15-8e86-1a045b7da0ed service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:18.631035 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1389/2780] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:07:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 16:07:20.504028 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600853de-f38f-484a-8954-7601628bd58e None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:20.504637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600853de-f38f-484a-8954-7601628bd58e None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:20.504760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600853de-f38f-484a-8954-7601628bd58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:20.536490 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-600853de-f38f-484a-8954-7601628bd58e None None] MFA Rules not processed for user `0f25bf41a74d43978435bcb856863b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:07:20.537747 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-600853de-f38f-484a-8954-7601628bd58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:20.608285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1392/2781] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:07:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:07:20.622659 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace8144-b4e2-4a15-8e86-1a045b7da0ed service nova] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:20.624050 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-4ace8144-b4e2-4a15-8e86-1a045b7da0ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:20.638590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace8144-b4e2-4a15-8e86-1a045b7da0ed service nova] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:20.644829 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058377e0-add9-4cd1-b0bd-c4f7fe9504d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:20.645483 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058377e0-add9-4cd1-b0bd-c4f7fe9504d3 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:20.645879 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058377e0-add9-4cd1-b0bd-c4f7fe9504d3 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:20.646032 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058377e0-add9-4cd1-b0bd-c4f7fe9504d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:20.655053 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058377e0-add9-4cd1-b0bd-c4f7fe9504d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:20.656713 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058377e0-add9-4cd1-b0bd-c4f7fe9504d3 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:20.666370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1390/2782] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:07:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:07:23.411750 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c425df-9e2d-438a-b6db-850d4f74f33c None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:23.412346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c425df-9e2d-438a-b6db-850d4f74f33c None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:23.412481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c425df-9e2d-438a-b6db-850d4f74f33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:23.431303 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-b5c425df-9e2d-438a-b6db-850d4f74f33c None None] MFA Rules not processed for user `0f25bf41a74d43978435bcb856863b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:07:23.432906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b5c425df-9e2d-438a-b6db-850d4f74f33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:23.445595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1393/2783] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:07:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:07:25.574419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058377e0-add9-4cd1-b0bd-c4f7fe9504d3 service nova] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:25.577901 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-058377e0-add9-4cd1-b0bd-c4f7fe9504d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:25.620935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:25.658918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058377e0-add9-4cd1-b0bd-c4f7fe9504d3 service nova] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:25.665475 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935617b-b9e7-40ee-88b5-7449021ef5de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:25.672729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935617b-b9e7-40ee-88b5-7449021ef5de service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:25.672729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935617b-b9e7-40ee-88b5-7449021ef5de service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:25.672729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935617b-b9e7-40ee-88b5-7449021ef5de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:25.672729 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f935617b-b9e7-40ee-88b5-7449021ef5de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:25.679965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935617b-b9e7-40ee-88b5-7449021ef5de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:25.681615 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935617b-b9e7-40ee-88b5-7449021ef5de service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:25.692795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1391/2784] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:07:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:07:30.185770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c425df-9e2d-438a-b6db-850d4f74f33c None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:30.187146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b5c425df-9e2d-438a-b6db-850d4f74f33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:30.199974 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c425df-9e2d-438a-b6db-850d4f74f33c None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:30.206618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18351213-f60f-45cc-bebd-9cfcabee7625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:30.206618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18351213-f60f-45cc-bebd-9cfcabee7625 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:30.206618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18351213-f60f-45cc-bebd-9cfcabee7625 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:30.206618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18351213-f60f-45cc-bebd-9cfcabee7625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:30.207044 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-18351213-f60f-45cc-bebd-9cfcabee7625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:30.218181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18351213-f60f-45cc-bebd-9cfcabee7625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:30.219608 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18351213-f60f-45cc-bebd-9cfcabee7625 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:30.229346 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1394/2785] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:07:34.073399 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935617b-b9e7-40ee-88b5-7449021ef5de service placement] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:34.074912 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f935617b-b9e7-40ee-88b5-7449021ef5de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:34.090537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935617b-b9e7-40ee-88b5-7449021ef5de service placement] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:34.095653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c995500-ceea-4c72-b663-8ff6c5d5da9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:34.096308 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c995500-ceea-4c72-b663-8ff6c5d5da9f service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:34.096631 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c995500-ceea-4c72-b663-8ff6c5d5da9f service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:34.096920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c995500-ceea-4c72-b663-8ff6c5d5da9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:34.098687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1c995500-ceea-4c72-b663-8ff6c5d5da9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:34.111844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c995500-ceea-4c72-b663-8ff6c5d5da9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:34.113573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c995500-ceea-4c72-b663-8ff6c5d5da9f service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:34.124285 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1392/2786] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:07:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:07:37.873846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18351213-f60f-45cc-bebd-9cfcabee7625 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:37.876138 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-18351213-f60f-45cc-bebd-9cfcabee7625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:37.891788 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18351213-f60f-45cc-bebd-9cfcabee7625 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:37.897983 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:37.898874 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:37.899182 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:37.899413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] PATH_INFO: `/v3/users/7e03ea067e27456590c0865766830d73` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:37.900691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e03ea067e27456590c0865766830d73)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:37.902669 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:37.917579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:38.060844 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e03ea067e27456590c0865766830d73'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:38.061491 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e03ea067e27456590c0865766830d73'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:38.062093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e03ea067e27456590c0865766830d73'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:38.070058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e03ea067e27456590c0865766830d73'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:38.087670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e03ea067e27456590c0865766830d73'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:38.196895 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1395/2787] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:07:37 2022] DELETE /identity/v3/users/7e03ea067e27456590c0865766830d73 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:07:38.445223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c995500-ceea-4c72-b663-8ff6c5d5da9f service placement] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:38.459147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c995500-ceea-4c72-b663-8ff6c5d5da9f service placement] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:38.532618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:38.533494 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:38.533783 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:38.534082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] PATH_INFO: `/v3/projects/8331debc88ed4f19b650426e1901b063` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:38.535795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8331debc88ed4f19b650426e1901b063)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:38.537749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:38.649602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8331debc88ed4f19b650426e1901b063', 'request_id': 'req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:38.655653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8331debc88ed4f19b650426e1901b063', 'request_id': 'req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:38.661627 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8331debc88ed4f19b650426e1901b063', 'request_id': 'req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:38.755252 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] The token cache is being invalidate because project 8331debc88ed4f19b650426e1901b063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:07:38.755552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:38.756220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:38.757815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1393/2788] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:07:38 2022] DELETE /identity/v3/projects/8331debc88ed4f19b650426e1901b063 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:07:38.761975 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:38.763413 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:38.776328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0a1bb-d7b1-446a-ae41-03c4ec58104e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:38.832203 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:38.833124 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:38.833406 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:38.833679 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] PATH_INFO: `/v3/users/a00b03687ca0447cab94d0015636602a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:38.834861 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a00b03687ca0447cab94d0015636602a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:38.836849 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:39.026931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a00b03687ca0447cab94d0015636602a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:39.027581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a00b03687ca0447cab94d0015636602a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:39.028082 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a00b03687ca0447cab94d0015636602a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:39.034812 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a00b03687ca0447cab94d0015636602a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:39.047404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a00b03687ca0447cab94d0015636602a'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:39.121411 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1396/2789] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:07:38 2022] DELETE /identity/v3/users/a00b03687ca0447cab94d0015636602a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:07:39.337599 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:39.339025 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:39.353650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada8d69-cd4a-4e4d-95e8-fbd8880b8831 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:39.396896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:39.411653 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:39.412549 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:39.412836 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:39.413111 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] PATH_INFO: `/v3/projects/f9069725e2a14eb9a412f50d226f36c7` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:39.414277 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9069725e2a14eb9a412f50d226f36c7)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:39.416167 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:39.547082 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9069725e2a14eb9a412f50d226f36c7', 'request_id': 'req-a195ece4-60d9-4b67-a505-0b660d79a171'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:39.553217 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9069725e2a14eb9a412f50d226f36c7', 'request_id': 'req-a195ece4-60d9-4b67-a505-0b660d79a171'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:39.558817 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9069725e2a14eb9a412f50d226f36c7', 'request_id': 'req-a195ece4-60d9-4b67-a505-0b660d79a171'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:39.650221 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] The token cache is being invalidate because project f9069725e2a14eb9a412f50d226f36c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:07:39.655749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:39.655749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-a195ece4-60d9-4b67-a505-0b660d79a171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:07:39.655749 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1394/2790] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:07:39 2022] DELETE /identity/v3/projects/f9069725e2a14eb9a412f50d226f36c7 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:07:39.975334 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:07:39.976836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:39.991756 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:40.052646 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bae503-8324-4081-8f9f-20554b03a3e4 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:07:40.116083 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8087e-301b-4690-b29f-82895fd5b63b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:07:40.116784 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8087e-301b-4690-b29f-82895fd5b63b service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:07:40.117074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8087e-301b-4690-b29f-82895fd5b63b service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:07:40.117359 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8087e-301b-4690-b29f-82895fd5b63b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:07:40.119191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4cd8087e-301b-4690-b29f-82895fd5b63b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:07:40.131306 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8087e-301b-4690-b29f-82895fd5b63b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:07:40.132945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8087e-301b-4690-b29f-82895fd5b63b service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:07:40.143319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1397/2791] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:07:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:07:49.646948 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:07:50.255721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=57 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:30.913430 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0619c1-8878-4f23-bc42-b6fb7eb62a3f None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:08:30.913938 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0619c1-8878-4f23-bc42-b6fb7eb62a3f None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:08:30.914066 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0619c1-8878-4f23-bc42-b6fb7eb62a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:08:30.942447 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-7d0619c1-8878-4f23-bc42-b6fb7eb62a3f None None] MFA Rules not processed for user `0f25bf41a74d43978435bcb856863b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:08:30.943685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7d0619c1-8878-4f23-bc42-b6fb7eb62a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:08:31.013426 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1395/2792] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:08:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:08:31.328743 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8087e-301b-4690-b29f-82895fd5b63b service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:08:31.329989 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4cd8087e-301b-4690-b29f-82895fd5b63b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:08:31.434765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8087e-301b-4690-b29f-82895fd5b63b service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:08:31.438540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bc415-3ec4-481f-8c8c-951ea3efd4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:08:31.439204 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bc415-3ec4-481f-8c8c-951ea3efd4ef service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:08:31.439489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bc415-3ec4-481f-8c8c-951ea3efd4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:08:31.439772 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bc415-3ec4-481f-8c8c-951ea3efd4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:08:31.441553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-741bc415-3ec4-481f-8c8c-951ea3efd4ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:08:31.451073 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bc415-3ec4-481f-8c8c-951ea3efd4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:08:31.452523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bc415-3ec4-481f-8c8c-951ea3efd4ef service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:08:31.461027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1398/2793] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:08:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:08:41.011326 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=37 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:08:41.457534 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=48 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:15.168043 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0619c1-8878-4f23-bc42-b6fb7eb62a3f None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:09:15.169695 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7d0619c1-8878-4f23-bc42-b6fb7eb62a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:09:15.271927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0619c1-8878-4f23-bc42-b6fb7eb62a3f None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:09:15.277169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d3c48-799f-4e3f-ad9d-48ed26f35f6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:09:15.277816 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3c48-799f-4e3f-ad9d-48ed26f35f6c service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:09:15.278096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3c48-799f-4e3f-ad9d-48ed26f35f6c service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:09:15.278807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3c48-799f-4e3f-ad9d-48ed26f35f6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:09:15.280621 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ee2d3c48-799f-4e3f-ad9d-48ed26f35f6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:09:15.292698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d3c48-799f-4e3f-ad9d-48ed26f35f6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:09:15.294268 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d3c48-799f-4e3f-ad9d-48ed26f35f6c service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:09:15.303951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1396/2794] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:09:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:09:25.299798 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=44 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:09:34.166873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47a513-7cd6-4fe6-a24a-10b1aa01c181 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:09:34.171187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47a513-7cd6-4fe6-a24a-10b1aa01c181 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:09:34.171187 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47a513-7cd6-4fe6-a24a-10b1aa01c181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:09:34.184893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ce47a513-7cd6-4fe6-a24a-10b1aa01c181 None None] MFA Rules not processed for user `0f25bf41a74d43978435bcb856863b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:09:34.186088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ce47a513-7cd6-4fe6-a24a-10b1aa01c181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:09:34.192912 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1399/2795] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:09:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:09:44.179957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:08.904484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce3dfe-875b-49ec-b38c-f248420db2b0 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:11:08.905034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce3dfe-875b-49ec-b38c-f248420db2b0 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:11:08.905034 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce3dfe-875b-49ec-b38c-f248420db2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:11:08.922787 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-c2ce3dfe-875b-49ec-b38c-f248420db2b0 None None] MFA Rules not processed for user `0f25bf41a74d43978435bcb856863b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:11:08.923916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-c2ce3dfe-875b-49ec-b38c-f248420db2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:11:08.930733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1397/2796] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:11:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:11:08.967424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47a513-7cd6-4fe6-a24a-10b1aa01c181 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:11:08.968843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ce47a513-7cd6-4fe6-a24a-10b1aa01c181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:11:09.046903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47a513-7cd6-4fe6-a24a-10b1aa01c181 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:11:09.050513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e408929-7007-41d9-9eb8-ef7be628e7b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '142e6e44d6544f6a891ed3f913de74ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:11:09.051166 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e408929-7007-41d9-9eb8-ef7be628e7b9 service glance] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:11:09.051444 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e408929-7007-41d9-9eb8-ef7be628e7b9 service glance] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:11:09.051717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e408929-7007-41d9-9eb8-ef7be628e7b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:11:09.053546 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3e408929-7007-41d9-9eb8-ef7be628e7b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:11:09.086214 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e408929-7007-41d9-9eb8-ef7be628e7b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:11:09.087643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e408929-7007-41d9-9eb8-ef7be628e7b9 service glance] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:11:09.107591 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1400/2797] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:11:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:11:09.791062 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce3dfe-875b-49ec-b38c-f248420db2b0 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:11:09.801077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce3dfe-875b-49ec-b38c-f248420db2b0 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:11:09.807530 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34ac4b-e8a7-43ea-bcd1-711c9b56bb4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:11:09.808202 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34ac4b-e8a7-43ea-bcd1-711c9b56bb4c service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:11:09.808530 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34ac4b-e8a7-43ea-bcd1-711c9b56bb4c service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:11:09.808920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34ac4b-e8a7-43ea-bcd1-711c9b56bb4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:11:09.810642 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7e34ac4b-e8a7-43ea-bcd1-711c9b56bb4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:11:09.825084 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34ac4b-e8a7-43ea-bcd1-711c9b56bb4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:11:09.826807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34ac4b-e8a7-43ea-bcd1-711c9b56bb4c service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:11:09.839406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1398/2798] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:11:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:11:19.104751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=47 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:11:19.834278 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=9 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:12.999727 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e408929-7007-41d9-9eb8-ef7be628e7b9 service glance] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:13.001327 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3e408929-7007-41d9-9eb8-ef7be628e7b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:13.022053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e408929-7007-41d9-9eb8-ef7be628e7b9 service glance] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:13.023738 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526b79a-d22a-45ce-b753-1bd20d9a2253 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:13.024522 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526b79a-d22a-45ce-b753-1bd20d9a2253 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:13.024877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526b79a-d22a-45ce-b753-1bd20d9a2253 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:13.025240 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526b79a-d22a-45ce-b753-1bd20d9a2253 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:13.027361 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2526b79a-d22a-45ce-b753-1bd20d9a2253 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:13.049946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526b79a-d22a-45ce-b753-1bd20d9a2253 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:13.055115 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526b79a-d22a-45ce-b753-1bd20d9a2253 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:13.064286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1401/2799] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:20.205875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34ac4b-e8a7-43ea-bcd1-711c9b56bb4c service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:20.207350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-7e34ac4b-e8a7-43ea-bcd1-711c9b56bb4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:20.221803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34ac4b-e8a7-43ea-bcd1-711c9b56bb4c service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:20.227865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d67db-1d08-4640-896b-6c5fba71f967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:20.228536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d67db-1d08-4640-896b-6c5fba71f967 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:20.228822 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d67db-1d08-4640-896b-6c5fba71f967 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:20.229096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d67db-1d08-4640-896b-6c5fba71f967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:20.230947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ab7d67db-1d08-4640-896b-6c5fba71f967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:20.243370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d67db-1d08-4640-896b-6c5fba71f967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:20.248442 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d67db-1d08-4640-896b-6c5fba71f967 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:20.255066 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1399/2800] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:20.296651 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526b79a-d22a-45ce-b753-1bd20d9a2253 service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:20.298108 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-2526b79a-d22a-45ce-b753-1bd20d9a2253 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:20.310088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526b79a-d22a-45ce-b753-1bd20d9a2253 service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:20.313771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5bd5f-e901-4864-a145-ebbc39f634b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:20.314414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5bd5f-e901-4864-a145-ebbc39f634b6 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:20.314694 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5bd5f-e901-4864-a145-ebbc39f634b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:20.314965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5bd5f-e901-4864-a145-ebbc39f634b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:20.316760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5fb5bd5f-e901-4864-a145-ebbc39f634b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:20.401062 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5bd5f-e901-4864-a145-ebbc39f634b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:20.402414 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5bd5f-e901-4864-a145-ebbc39f634b6 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:20.421779 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1402/2801] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:23.837609 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d67db-1d08-4640-896b-6c5fba71f967 service nova] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:23.847571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d67db-1d08-4640-896b-6c5fba71f967 service nova] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:23.856497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4b20d-c4de-44f2-bfc4-167b27cf1274 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:23.857605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4b20d-c4de-44f2-bfc4-167b27cf1274 service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:23.857976 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4b20d-c4de-44f2-bfc4-167b27cf1274 service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:23.858335 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4b20d-c4de-44f2-bfc4-167b27cf1274 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:23.861164 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-65f4b20d-c4de-44f2-bfc4-167b27cf1274 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:23.879751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4b20d-c4de-44f2-bfc4-167b27cf1274 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:23.883239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4b20d-c4de-44f2-bfc4-167b27cf1274 service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:23.898523 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1400/2802] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:23.959036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5bd5f-e901-4864-a145-ebbc39f634b6 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:23.960285 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-5fb5bd5f-e901-4864-a145-ebbc39f634b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:23.973948 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5bd5f-e901-4864-a145-ebbc39f634b6 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:23.978946 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:23.980019 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:23.980543 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:23.980746 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] PATH_INFO: `/v3/users/0f25bf41a74d43978435bcb856863b5b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:23.982039 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f25bf41a74d43978435bcb856863b5b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:23.984604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:24.153202 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f25bf41a74d43978435bcb856863b5b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:24.153836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f25bf41a74d43978435bcb856863b5b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:24.154479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f25bf41a74d43978435bcb856863b5b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:24.161869 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f25bf41a74d43978435bcb856863b5b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:24.173343 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ec7fac5b-7b3a-405b-861c-23db9d1cb092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f25bf41a74d43978435bcb856863b5b'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:24.256301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1403/2803] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:12:23 2022] DELETE /identity/v3/users/0f25bf41a74d43978435bcb856863b5b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:24.484100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4b20d-c4de-44f2-bfc4-167b27cf1274 service placement] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:24.495733 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4b20d-c4de-44f2-bfc4-167b27cf1274 service placement] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:24.553316 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:24.554738 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:24.555022 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:24.555298 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] PATH_INFO: `/v3/projects/f7e46ae2884d4ae48592abdd6569e499` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:24.556532 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e46ae2884d4ae48592abdd6569e499)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:24.558436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:24.682085 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e46ae2884d4ae48592abdd6569e499', 'request_id': 'req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:24.688727 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e46ae2884d4ae48592abdd6569e499', 'request_id': 'req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:24.695954 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e46ae2884d4ae48592abdd6569e499', 'request_id': 'req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:24.749865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] The token cache is being invalidate because project f7e46ae2884d4ae48592abdd6569e499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87074) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:12:24.755360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:24.755360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ca519a70-4159-41d0-bf2e-cedefc6ccb9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:24.755360 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1401/2804] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:12:24 2022] DELETE /identity/v3/projects/f7e46ae2884d4ae48592abdd6569e499 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:24.757483 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aaecb-032a-4f6e-a678-58d5a2499796 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:24.757770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aaecb-032a-4f6e-a678-58d5a2499796 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:24.758047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aaecb-032a-4f6e-a678-58d5a2499796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:24.791102 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:24.816333 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-ec6aaecb-032a-4f6e-a678-58d5a2499796 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:24.817573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-ec6aaecb-032a-4f6e-a678-58d5a2499796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:24.876634 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1404/2805] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:24.884456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909264-c749-4cdc-97a1-4308debe8bbf None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:24.884949 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909264-c749-4cdc-97a1-4308debe8bbf None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:24.885058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c909264-c749-4cdc-97a1-4308debe8bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:24.904558 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-4c909264-c749-4cdc-97a1-4308debe8bbf None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:24.905784 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-4c909264-c749-4cdc-97a1-4308debe8bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:24.914576 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1402/2806] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:24.919629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aaecb-032a-4f6e-a678-58d5a2499796 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:24.930883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aaecb-032a-4f6e-a678-58d5a2499796 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:24.935654 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e72e8-a367-4df7-8d26-6f430ffb4d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:24.936635 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e72e8-a367-4df7-8d26-6f430ffb4d78 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:24.936918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e72e8-a367-4df7-8d26-6f430ffb4d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:24.937191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e72e8-a367-4df7-8d26-6f430ffb4d78 admin admin] PATH_INFO: `/v3/domains` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:24.937918 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e72e8-a367-4df7-8d26-6f430ffb4d78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87073) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:12:24.938251 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e72e8-a367-4df7-8d26-6f430ffb4d78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:24.939883 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e72e8-a367-4df7-8d26-6f430ffb4d78 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:24.951690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1405/2807] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:12:24.956248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c909264-c749-4cdc-97a1-4308debe8bbf None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:24.966684 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c909264-c749-4cdc-97a1-4308debe8bbf None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:24.973382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16b71b-904a-4ed6-bd21-339a31e5484d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:24.973382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16b71b-904a-4ed6-bd21-339a31e5484d admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:24.973382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16b71b-904a-4ed6-bd21-339a31e5484d admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:24.973382 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16b71b-904a-4ed6-bd21-339a31e5484d admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:24.973789 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16b71b-904a-4ed6-bd21-339a31e5484d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:24.975201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16b71b-904a-4ed6-bd21-339a31e5484d admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:25.066832 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1403/2808] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:12:24 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:25.070933 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e72e8-a367-4df7-8d26-6f430ffb4d78 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:25.081173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e72e8-a367-4df7-8d26-6f430ffb4d78 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:25.137286 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753857-beea-4f1b-a450-82a36d804a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:25.137956 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753857-beea-4f1b-a450-82a36d804a2a admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:25.138241 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753857-beea-4f1b-a450-82a36d804a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:25.138520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753857-beea-4f1b-a450-82a36d804a2a admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:25.139199 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c753857-beea-4f1b-a450-82a36d804a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:25.141155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c753857-beea-4f1b-a450-82a36d804a2a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:25.251319 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1406/2809] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:12:25 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:25.255385 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16b71b-904a-4ed6-bd21-339a31e5484d admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:25.265736 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16b71b-904a-4ed6-bd21-339a31e5484d admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:25.270280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbca9c-1264-46a8-a1b5-1d6e0bda8ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:25.270936 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbca9c-1264-46a8-a1b5-1d6e0bda8ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:25.271222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbca9c-1264-46a8-a1b5-1d6e0bda8ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:25.271501 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbca9c-1264-46a8-a1b5-1d6e0bda8ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:25.272807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedbca9c-1264-46a8-a1b5-1d6e0bda8ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:25.274373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedbca9c-1264-46a8-a1b5-1d6e0bda8ce9 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:25.284708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1404/2810] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:25.289367 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753857-beea-4f1b-a450-82a36d804a2a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:25.299624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753857-beea-4f1b-a450-82a36d804a2a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:25.304239 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20d508-92aa-45da-93c7-ffca86c09836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:25.304941 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20d508-92aa-45da-93c7-ffca86c09836 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:25.305215 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20d508-92aa-45da-93c7-ffca86c09836 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:25.305490 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20d508-92aa-45da-93c7-ffca86c09836 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:25.306211 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20d508-92aa-45da-93c7-ffca86c09836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:25.307859 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20d508-92aa-45da-93c7-ffca86c09836 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:25.317742 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1407/2811] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:25.322263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbca9c-1264-46a8-a1b5-1d6e0bda8ce9 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:25.332105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbca9c-1264-46a8-a1b5-1d6e0bda8ce9 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:25.337581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5c6e4-3ddc-44c0-af0b-58871eb39faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:25.338468 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5c6e4-3ddc-44c0-af0b-58871eb39faa admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:25.338751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5c6e4-3ddc-44c0-af0b-58871eb39faa admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:25.339027 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5c6e4-3ddc-44c0-af0b-58871eb39faa admin admin] PATH_INFO: `/v3/projects/10ec7529e0db4abdb33dcbdf394726b9/users/114a8332cc014c418e36c5f9ecfeb1f3/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:25.353751 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5c6e4-3ddc-44c0-af0b-58871eb39faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ec7529e0db4abdb33dcbdf394726b9, user_id=114a8332cc014c418e36c5f9ecfeb1f3, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:25.356448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5c6e4-3ddc-44c0-af0b-58871eb39faa admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:25.472101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:25.476924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1405/2812] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:12:25 2022] PUT /identity/v3/projects/10ec7529e0db4abdb33dcbdf394726b9/users/114a8332cc014c418e36c5f9ecfeb1f3/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:25.488495 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c13b27-f080-4d6f-a5dd-7d2eb85d4630 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:25.488791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c13b27-f080-4d6f-a5dd-7d2eb85d4630 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:25.489220 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c13b27-f080-4d6f-a5dd-7d2eb85d4630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:25.517198 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-81c13b27-f080-4d6f-a5dd-7d2eb85d4630 None None] MFA Rules not processed for user `114a8332cc014c418e36c5f9ecfeb1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:25.518494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-81c13b27-f080-4d6f-a5dd-7d2eb85d4630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:25.564832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:25.582852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1408/2813] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:25.587958 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5c6e4-3ddc-44c0-af0b-58871eb39faa admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:25.650605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5c6e4-3ddc-44c0-af0b-58871eb39faa admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:25.655257 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8cf7c-f112-4bc2-9ba0-ce58f15c4a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:25.655923 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8cf7c-f112-4bc2-9ba0-ce58f15c4a8e admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:25.656234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8cf7c-f112-4bc2-9ba0-ce58f15c4a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:25.656550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8cf7c-f112-4bc2-9ba0-ce58f15c4a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:25.657260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8cf7c-f112-4bc2-9ba0-ce58f15c4a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:25.659170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8cf7c-f112-4bc2-9ba0-ce58f15c4a8e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:25.758142 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1406/2814] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:12:25 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:25.762365 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c13b27-f080-4d6f-a5dd-7d2eb85d4630 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:25.773153 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c13b27-f080-4d6f-a5dd-7d2eb85d4630 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:25.830373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae221c-d264-497d-933b-349c9b53eac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:25.831074 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae221c-d264-497d-933b-349c9b53eac6 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:25.831357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae221c-d264-497d-933b-349c9b53eac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:25.831629 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae221c-d264-497d-933b-349c9b53eac6 admin admin] PATH_INFO: `/v3/users` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:25.832548 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae221c-d264-497d-933b-349c9b53eac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:25.834513 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae221c-d264-497d-933b-349c9b53eac6 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:25.957595 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1409/2815] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:12:25 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:25.961482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8cf7c-f112-4bc2-9ba0-ce58f15c4a8e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:25.971960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8cf7c-f112-4bc2-9ba0-ce58f15c4a8e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:25.976679 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4fb69-7702-4949-a5f8-fcdd919d0a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:25.977345 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4fb69-7702-4949-a5f8-fcdd919d0a78 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:25.977637 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4fb69-7702-4949-a5f8-fcdd919d0a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:25.977915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4fb69-7702-4949-a5f8-fcdd919d0a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:25.978643 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4fb69-7702-4949-a5f8-fcdd919d0a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:25.980225 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4fb69-7702-4949-a5f8-fcdd919d0a78 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:25.991581 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1407/2816] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:26.001657 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae221c-d264-497d-933b-349c9b53eac6 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:26.012920 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae221c-d264-497d-933b-349c9b53eac6 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:26.017666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df889f88-3242-4b39-b5e9-8ad3863aa61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:26.018558 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df889f88-3242-4b39-b5e9-8ad3863aa61a admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:26.018843 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df889f88-3242-4b39-b5e9-8ad3863aa61a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:26.019120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df889f88-3242-4b39-b5e9-8ad3863aa61a admin admin] PATH_INFO: `/v3/projects/42e452deabf54eef8e4fb362853c2f04/users/8d92bdeb3635477681540abfe608c19e/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:26.035730 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df889f88-3242-4b39-b5e9-8ad3863aa61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e452deabf54eef8e4fb362853c2f04, user_id=8d92bdeb3635477681540abfe608c19e, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:26.038957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df889f88-3242-4b39-b5e9-8ad3863aa61a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:26.128224 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1410/2817] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:12:26 2022] PUT /identity/v3/projects/42e452deabf54eef8e4fb362853c2f04/users/8d92bdeb3635477681540abfe608c19e/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:26.132439 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4fb69-7702-4949-a5f8-fcdd919d0a78 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:26.142737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4fb69-7702-4949-a5f8-fcdd919d0a78 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:26.199645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f623b-ff8c-4261-9f9f-54c93a6ae270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:26.200317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f623b-ff8c-4261-9f9f-54c93a6ae270 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:26.200645 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f623b-ff8c-4261-9f9f-54c93a6ae270 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:26.200924 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f623b-ff8c-4261-9f9f-54c93a6ae270 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:26.201657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f623b-ff8c-4261-9f9f-54c93a6ae270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:26.203248 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f623b-ff8c-4261-9f9f-54c93a6ae270 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:26.213683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1408/2818] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:26.217961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df889f88-3242-4b39-b5e9-8ad3863aa61a admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:26.228212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df889f88-3242-4b39-b5e9-8ad3863aa61a admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:26.233091 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2807ae9-4358-4144-8d48-cf430c4d8cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:26.233965 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2807ae9-4358-4144-8d48-cf430c4d8cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:26.234249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2807ae9-4358-4144-8d48-cf430c4d8cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:26.234523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2807ae9-4358-4144-8d48-cf430c4d8cb3 admin admin] PATH_INFO: `/v3/projects/42e452deabf54eef8e4fb362853c2f04/users/8d92bdeb3635477681540abfe608c19e/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:26.236511 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2807ae9-4358-4144-8d48-cf430c4d8cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e452deabf54eef8e4fb362853c2f04, user_id=8d92bdeb3635477681540abfe608c19e, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:26.239117 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2807ae9-4358-4144-8d48-cf430c4d8cb3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:26.249707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1411/2819] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:12:26 2022] PUT /identity/v3/projects/42e452deabf54eef8e4fb362853c2f04/users/8d92bdeb3635477681540abfe608c19e/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:26.260373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1b9a5-2409-4f13-b2db-78574ea6d633 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:26.260687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1b9a5-2409-4f13-b2db-78574ea6d633 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:26.261002 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1b9a5-2409-4f13-b2db-78574ea6d633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:26.288350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-dcb1b9a5-2409-4f13-b2db-78574ea6d633 None None] MFA Rules not processed for user `8d92bdeb3635477681540abfe608c19e`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:26.289650 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dcb1b9a5-2409-4f13-b2db-78574ea6d633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:26.347702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:26.356263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1409/2820] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:26.367131 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2807ae9-4358-4144-8d48-cf430c4d8cb3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:26.368932 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c2807ae9-4358-4144-8d48-cf430c4d8cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:26.448209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2807ae9-4358-4144-8d48-cf430c4d8cb3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:26.452264 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab093021-faaf-42f3-87e1-746aef11d93c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:26.452945 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab093021-faaf-42f3-87e1-746aef11d93c service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:26.453234 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab093021-faaf-42f3-87e1-746aef11d93c service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:26.453515 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab093021-faaf-42f3-87e1-746aef11d93c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:26.462771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab093021-faaf-42f3-87e1-746aef11d93c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:26.464212 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab093021-faaf-42f3-87e1-746aef11d93c service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:26.473925 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1412/2821] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:26 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:29.160386 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1b9a5-2409-4f13-b2db-78574ea6d633 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:29.223606 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1b9a5-2409-4f13-b2db-78574ea6d633 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:29.228708 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:29.229577 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:29.229863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:29.230143 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] PATH_INFO: `/v3/users/114a8332cc014c418e36c5f9ecfeb1f3` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:29.231377 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] RBAC: Authorizing `identity:delete_user(user_id=114a8332cc014c418e36c5f9ecfeb1f3)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:29.233529 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:29.413689 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114a8332cc014c418e36c5f9ecfeb1f3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:29.414352 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114a8332cc014c418e36c5f9ecfeb1f3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:29.426639 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114a8332cc014c418e36c5f9ecfeb1f3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:29.427284 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114a8332cc014c418e36c5f9ecfeb1f3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:29.434115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114a8332cc014c418e36c5f9ecfeb1f3'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:29.513496 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1410/2822] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:12:29 2022] DELETE /identity/v3/users/114a8332cc014c418e36c5f9ecfeb1f3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:29.518132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500fedb-f11c-44a6-9eae-91ff540ae355 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:29.518597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500fedb-f11c-44a6-9eae-91ff540ae355 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:29.518697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500fedb-f11c-44a6-9eae-91ff540ae355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:29.538250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-f500fedb-f11c-44a6-9eae-91ff540ae355 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:29.540018 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f500fedb-f11c-44a6-9eae-91ff540ae355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:29.558098 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:29.598490 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1413/2823] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:12:29.608888 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:29.610280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:29.666573 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18699272-e9fa-4680-bd12-c1b04291656b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:29.670175 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a2e35-0267-4e4f-8972-ce566d280443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:29.670820 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a2e35-0267-4e4f-8972-ce566d280443 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:29.671100 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a2e35-0267-4e4f-8972-ce566d280443 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:29.671372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a2e35-0267-4e4f-8972-ce566d280443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:29.680456 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a2e35-0267-4e4f-8972-ce566d280443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:29.681875 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a2e35-0267-4e4f-8972-ce566d280443 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:29.691359 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1411/2824] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:30.095355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500fedb-f11c-44a6-9eae-91ff540ae355 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:30.096818 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-f500fedb-f11c-44a6-9eae-91ff540ae355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:30.111315 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500fedb-f11c-44a6-9eae-91ff540ae355 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:30.115997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:30.117926 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:30.118217 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:30.118494 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] PATH_INFO: `/v3/projects/10ec7529e0db4abdb33dcbdf394726b9` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:30.119687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10ec7529e0db4abdb33dcbdf394726b9)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:30.121705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:30.217777 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10ec7529e0db4abdb33dcbdf394726b9', 'request_id': 'req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:30.223893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10ec7529e0db4abdb33dcbdf394726b9', 'request_id': 'req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:30.229419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10ec7529e0db4abdb33dcbdf394726b9', 'request_id': 'req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:30.304128 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] The token cache is being invalidate because project 10ec7529e0db4abdb33dcbdf394726b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:12:30.305021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:30.305650 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:30.307408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1414/2825] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:12:30 2022] DELETE /identity/v3/projects/10ec7529e0db4abdb33dcbdf394726b9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:30.311435 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a2e35-0267-4e4f-8972-ce566d280443 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:30.321657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a2e35-0267-4e4f-8972-ce566d280443 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:30.333544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:30.377918 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:30.378711 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:30.378992 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:30.379267 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] PATH_INFO: `/v3/users/8d92bdeb3635477681540abfe608c19e` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:30.380484 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d92bdeb3635477681540abfe608c19e)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:30.383040 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:30.553023 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d92bdeb3635477681540abfe608c19e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:30.553702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d92bdeb3635477681540abfe608c19e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:30.566436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d92bdeb3635477681540abfe608c19e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:30.567105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d92bdeb3635477681540abfe608c19e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:30.573969 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ef96186e-3d14-4f22-9c5c-36fc6c1e1c0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d92bdeb3635477681540abfe608c19e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:30.653429 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1412/2826] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:12:30 2022] DELETE /identity/v3/users/8d92bdeb3635477681540abfe608c19e => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:31.003763 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:31.007460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:31.018767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1faf2-57d4-40bc-9b0c-474463c1a8e8 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:31.077317 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:31.078236 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:31.078517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:31.078790 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] PATH_INFO: `/v3/projects/42e452deabf54eef8e4fb362853c2f04` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:31.079955 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e452deabf54eef8e4fb362853c2f04)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:31.081931 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:31.168691 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e452deabf54eef8e4fb362853c2f04', 'request_id': 'req-ab69fc16-af8f-4ffb-9280-89599a1e24c2'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:31.169700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:31.175394 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e452deabf54eef8e4fb362853c2f04', 'request_id': 'req-ab69fc16-af8f-4ffb-9280-89599a1e24c2'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:31.180970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e452deabf54eef8e4fb362853c2f04', 'request_id': 'req-ab69fc16-af8f-4ffb-9280-89599a1e24c2'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:31.255953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] The token cache is being invalidate because project 42e452deabf54eef8e4fb362853c2f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:12:31.256243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:31.256896 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-ab69fc16-af8f-4ffb-9280-89599a1e24c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:31.258953 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1415/2827] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:12:31 2022] DELETE /identity/v3/projects/42e452deabf54eef8e4fb362853c2f04 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:12:31.263943 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d91b9-0242-4ff3-b339-65cd9ba6d606 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:31.264240 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d91b9-0242-4ff3-b339-65cd9ba6d606 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:31.264564 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d91b9-0242-4ff3-b339-65cd9ba6d606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:31.282863 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-225d91b9-0242-4ff3-b339-65cd9ba6d606 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:31.284074 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-225d91b9-0242-4ff3-b339-65cd9ba6d606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:31.344878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1413/2828] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:31.352707 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686481b-9bcb-448b-ac50-867da47970ca None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:31.353006 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686481b-9bcb-448b-ac50-867da47970ca None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:31.353289 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686481b-9bcb-448b-ac50-867da47970ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:31.371908 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-3686481b-9bcb-448b-ac50-867da47970ca None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:31.373154 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-3686481b-9bcb-448b-ac50-867da47970ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:31.380947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1416/2829] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 16:12:31.385723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d91b9-0242-4ff3-b339-65cd9ba6d606 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:31.395220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d91b9-0242-4ff3-b339-65cd9ba6d606 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:31.400657 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a778dc-3376-4831-b913-e200c8f9b778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:31.401314 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a778dc-3376-4831-b913-e200c8f9b778 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:31.401595 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a778dc-3376-4831-b913-e200c8f9b778 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:31.401891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a778dc-3376-4831-b913-e200c8f9b778 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:31.402616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a778dc-3376-4831-b913-e200c8f9b778 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:12:31.402920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a778dc-3376-4831-b913-e200c8f9b778 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:31.404534 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a778dc-3376-4831-b913-e200c8f9b778 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:31.416075 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1414/2830] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:12:31.420520 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686481b-9bcb-448b-ac50-867da47970ca None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:31.429692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686481b-9bcb-448b-ac50-867da47970ca None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:31.434316 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55250cd-b310-4520-beb1-9a57250102eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:31.434962 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55250cd-b310-4520-beb1-9a57250102eb admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:31.435249 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55250cd-b310-4520-beb1-9a57250102eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:31.435530 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55250cd-b310-4520-beb1-9a57250102eb admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:31.436230 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55250cd-b310-4520-beb1-9a57250102eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:31.439001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55250cd-b310-4520-beb1-9a57250102eb admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:31.552060 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1417/2831] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:12:31 2022] POST /identity/v3/projects => generated 401 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:31.556292 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a778dc-3376-4831-b913-e200c8f9b778 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:31.566370 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a778dc-3376-4831-b913-e200c8f9b778 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:31.588239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:31.623396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb55371-e894-4d9d-9b71-0350ea2825c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:31.624073 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb55371-e894-4d9d-9b71-0350ea2825c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:31.624354 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb55371-e894-4d9d-9b71-0350ea2825c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:31.624656 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb55371-e894-4d9d-9b71-0350ea2825c9 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:31.625348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb55371-e894-4d9d-9b71-0350ea2825c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:31.627239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb55371-e894-4d9d-9b71-0350ea2825c9 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:31.756691 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1415/2832] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:12:31 2022] POST /identity/v3/users => generated 301 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:31.761544 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55250cd-b310-4520-beb1-9a57250102eb admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:31.771865 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55250cd-b310-4520-beb1-9a57250102eb admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:31.776597 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98a44f-5f29-4505-b5d5-11ca4aa5cbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:31.777256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a44f-5f29-4505-b5d5-11ca4aa5cbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:31.777637 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a44f-5f29-4505-b5d5-11ca4aa5cbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:31.777917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a44f-5f29-4505-b5d5-11ca4aa5cbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:31.778685 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98a44f-5f29-4505-b5d5-11ca4aa5cbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:31.780283 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98a44f-5f29-4505-b5d5-11ca4aa5cbe5 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:31.790550 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1418/2833] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:31.794706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb55371-e894-4d9d-9b71-0350ea2825c9 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:31.805550 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb55371-e894-4d9d-9b71-0350ea2825c9 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:31.810086 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fece4-c5fc-444d-b652-34fb58bf261e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:31.810731 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fece4-c5fc-444d-b652-34fb58bf261e admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:31.811013 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fece4-c5fc-444d-b652-34fb58bf261e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:31.811289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fece4-c5fc-444d-b652-34fb58bf261e admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:31.812005 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fece4-c5fc-444d-b652-34fb58bf261e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:31.813613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fece4-c5fc-444d-b652-34fb58bf261e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:31.824582 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1416/2834] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:31.828809 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98a44f-5f29-4505-b5d5-11ca4aa5cbe5 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:31.838786 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98a44f-5f29-4505-b5d5-11ca4aa5cbe5 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:31.843244 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6120b-5dbd-4edf-b3cf-d30b85f2038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:31.844118 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6120b-5dbd-4edf-b3cf-d30b85f2038d admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:31.844424 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6120b-5dbd-4edf-b3cf-d30b85f2038d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:31.844704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6120b-5dbd-4edf-b3cf-d30b85f2038d admin admin] PATH_INFO: `/v3/projects/27f8a5a13c3840f587832ae5bc313301/users/a0d1fc7252fe46049067eb4932d1029c/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:31.859497 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6120b-5dbd-4edf-b3cf-d30b85f2038d admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f8a5a13c3840f587832ae5bc313301, user_id=a0d1fc7252fe46049067eb4932d1029c, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:31.862161 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6120b-5dbd-4edf-b3cf-d30b85f2038d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:31.939579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1419/2835] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:12:31 2022] PUT /identity/v3/projects/27f8a5a13c3840f587832ae5bc313301/users/a0d1fc7252fe46049067eb4932d1029c/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:31.999887 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2122b-15a6-4d7a-9f5e-9428a10a7069 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:32.000161 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2122b-15a6-4d7a-9f5e-9428a10a7069 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:32.000480 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2122b-15a6-4d7a-9f5e-9428a10a7069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:32.027935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-6cf2122b-15a6-4d7a-9f5e-9428a10a7069 None None] MFA Rules not processed for user `a0d1fc7252fe46049067eb4932d1029c`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:32.029102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6cf2122b-15a6-4d7a-9f5e-9428a10a7069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:32.093012 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1417/2836] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:32.097542 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6120b-5dbd-4edf-b3cf-d30b85f2038d admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:32.157991 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6120b-5dbd-4edf-b3cf-d30b85f2038d admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:32.162614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96abac-0ab3-42bd-98a0-c99d4a3df965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:32.163268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96abac-0ab3-42bd-98a0-c99d4a3df965 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:32.163551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96abac-0ab3-42bd-98a0-c99d4a3df965 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:32.164330 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96abac-0ab3-42bd-98a0-c99d4a3df965 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:32.165053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96abac-0ab3-42bd-98a0-c99d4a3df965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:32.166934 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96abac-0ab3-42bd-98a0-c99d4a3df965 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:32.232245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1420/2837] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:12:32 2022] POST /identity/v3/projects => generated 399 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:32.236604 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2122b-15a6-4d7a-9f5e-9428a10a7069 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:32.248218 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2122b-15a6-4d7a-9f5e-9428a10a7069 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:32.305223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a1338-8e43-45f6-a238-50bdb151f230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:32.305904 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a1338-8e43-45f6-a238-50bdb151f230 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:32.306191 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a1338-8e43-45f6-a238-50bdb151f230 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:32.306471 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a1338-8e43-45f6-a238-50bdb151f230 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:32.307163 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a1338-8e43-45f6-a238-50bdb151f230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:32.309121 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a1338-8e43-45f6-a238-50bdb151f230 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:32.506071 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1418/2838] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:12:32 2022] POST /identity/v3/users => generated 306 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:32.509802 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96abac-0ab3-42bd-98a0-c99d4a3df965 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:32.519968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96abac-0ab3-42bd-98a0-c99d4a3df965 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:32.525076 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7975b-db68-49c4-8922-faff2f2da56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:32.525739 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7975b-db68-49c4-8922-faff2f2da56d admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:32.526021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7975b-db68-49c4-8922-faff2f2da56d admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:32.526325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7975b-db68-49c4-8922-faff2f2da56d admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:32.527047 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7975b-db68-49c4-8922-faff2f2da56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:32.528666 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7975b-db68-49c4-8922-faff2f2da56d admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:32.540770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1421/2839] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:32.544891 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a1338-8e43-45f6-a238-50bdb151f230 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:32.554805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a1338-8e43-45f6-a238-50bdb151f230 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:32.559344 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c000ad0-cc9b-410d-991a-f7bc3a850bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:32.560220 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c000ad0-cc9b-410d-991a-f7bc3a850bef admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:32.560544 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c000ad0-cc9b-410d-991a-f7bc3a850bef admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:32.560824 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c000ad0-cc9b-410d-991a-f7bc3a850bef admin admin] PATH_INFO: `/v3/projects/205ba6f91b9747e8b8e0bbf791b3034a/users/15ed89d88f934365b156f17fbeb22c44/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:32.575628 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c000ad0-cc9b-410d-991a-f7bc3a850bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=205ba6f91b9747e8b8e0bbf791b3034a, user_id=15ed89d88f934365b156f17fbeb22c44, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:32.578283 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c000ad0-cc9b-410d-991a-f7bc3a850bef admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:32.630930 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:32.636518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1419/2840] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:12:32 2022] PUT /identity/v3/projects/205ba6f91b9747e8b8e0bbf791b3034a/users/15ed89d88f934365b156f17fbeb22c44/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:32.640653 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7975b-db68-49c4-8922-faff2f2da56d admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:32.650973 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7975b-db68-49c4-8922-faff2f2da56d admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:32.694846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:32.705887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb9b38-4370-425e-b624-91cdc9d8845a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:32.706557 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb9b38-4370-425e-b624-91cdc9d8845a admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:32.706842 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb9b38-4370-425e-b624-91cdc9d8845a admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:32.707122 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb9b38-4370-425e-b624-91cdc9d8845a admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:32.707836 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb9b38-4370-425e-b624-91cdc9d8845a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:32.709451 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb9b38-4370-425e-b624-91cdc9d8845a admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:32.719697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1422/2841] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:32.723885 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c000ad0-cc9b-410d-991a-f7bc3a850bef admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:32.735284 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c000ad0-cc9b-410d-991a-f7bc3a850bef admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:32.740001 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc941b-a1a6-4be6-a389-1f592ce7c8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:32.740940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc941b-a1a6-4be6-a389-1f592ce7c8c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:32.741223 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc941b-a1a6-4be6-a389-1f592ce7c8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:32.741505 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc941b-a1a6-4be6-a389-1f592ce7c8c3 admin admin] PATH_INFO: `/v3/projects/205ba6f91b9747e8b8e0bbf791b3034a/users/15ed89d88f934365b156f17fbeb22c44/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:32.743508 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc941b-a1a6-4be6-a389-1f592ce7c8c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=205ba6f91b9747e8b8e0bbf791b3034a, user_id=15ed89d88f934365b156f17fbeb22c44, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:32.746151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc941b-a1a6-4be6-a389-1f592ce7c8c3 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:32.756541 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1420/2842] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:12:32 2022] PUT /identity/v3/projects/205ba6f91b9747e8b8e0bbf791b3034a/users/15ed89d88f934365b156f17fbeb22c44/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:32.767146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d822486-44b8-40c7-b2c0-ae129f75e955 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:32.767429 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d822486-44b8-40c7-b2c0-ae129f75e955 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:32.767700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d822486-44b8-40c7-b2c0-ae129f75e955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:32.794997 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0d822486-44b8-40c7-b2c0-ae129f75e955 None None] MFA Rules not processed for user `15ed89d88f934365b156f17fbeb22c44`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:32.796146 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0d822486-44b8-40c7-b2c0-ae129f75e955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:32.863556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1423/2843] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:32 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:32.874243 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc941b-a1a6-4be6-a389-1f592ce7c8c3 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:32.875934 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5cdc941b-a1a6-4be6-a389-1f592ce7c8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:32.932509 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc941b-a1a6-4be6-a389-1f592ce7c8c3 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:32.936247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc185d4-3456-4b21-a07b-081da99c952e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:32.936932 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc185d4-3456-4b21-a07b-081da99c952e service cinder] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:32.937215 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc185d4-3456-4b21-a07b-081da99c952e service cinder] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:32.937493 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc185d4-3456-4b21-a07b-081da99c952e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:32.947126 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc185d4-3456-4b21-a07b-081da99c952e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:32.951630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc185d4-3456-4b21-a07b-081da99c952e service cinder] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:32.958109 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1421/2844] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:32 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:33.200508 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d822486-44b8-40c7-b2c0-ae129f75e955 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:33.210180 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d822486-44b8-40c7-b2c0-ae129f75e955 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:33.213846 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee3a88-4d0c-4873-98eb-7981af6da0ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:33.214489 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee3a88-4d0c-4873-98eb-7981af6da0ef service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:33.214770 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee3a88-4d0c-4873-98eb-7981af6da0ef service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:33.215046 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee3a88-4d0c-4873-98eb-7981af6da0ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:33.272586 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee3a88-4d0c-4873-98eb-7981af6da0ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:33.274031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee3a88-4d0c-4873-98eb-7981af6da0ef service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:33.283115 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1424/2845] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:38.197234 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc185d4-3456-4b21-a07b-081da99c952e service cinder] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:38.259051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc185d4-3456-4b21-a07b-081da99c952e service cinder] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:38.263685 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:38.264588 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:38.264878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:38.265155 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] PATH_INFO: `/v3/users/a0d1fc7252fe46049067eb4932d1029c` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:38.266348 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d1fc7252fe46049067eb4932d1029c)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:38.268289 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:38.457986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d1fc7252fe46049067eb4932d1029c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:38.458572 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d1fc7252fe46049067eb4932d1029c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:38.470737 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d1fc7252fe46049067eb4932d1029c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:38.471365 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d1fc7252fe46049067eb4932d1029c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:38.478986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d1fc7252fe46049067eb4932d1029c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:38.549692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1422/2846] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:12:38 2022] DELETE /identity/v3/users/a0d1fc7252fe46049067eb4932d1029c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:38.554231 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd2b25-5a18-407c-9406-7c17b2178cf2 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:38.554728 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd2b25-5a18-407c-9406-7c17b2178cf2 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:38.554839 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd2b25-5a18-407c-9406-7c17b2178cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:38.572065 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-0cbd2b25-5a18-407c-9406-7c17b2178cf2 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:38.573290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0cbd2b25-5a18-407c-9406-7c17b2178cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:38.631662 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1425/2847] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:38.641607 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:38.642986 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:38.653389 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:38.700461 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8073956-a94a-4cc6-af59-f847ae35e77e admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:38.704101 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83ae58-665f-4636-a31d-cfa5baf7d7b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:38.704776 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83ae58-665f-4636-a31d-cfa5baf7d7b2 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:38.705057 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83ae58-665f-4636-a31d-cfa5baf7d7b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:38.705330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83ae58-665f-4636-a31d-cfa5baf7d7b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:38.714967 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83ae58-665f-4636-a31d-cfa5baf7d7b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:38.717212 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83ae58-665f-4636-a31d-cfa5baf7d7b2 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:38.727399 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1423/2848] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:39.189245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd2b25-5a18-407c-9406-7c17b2178cf2 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:39.190681 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-0cbd2b25-5a18-407c-9406-7c17b2178cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:39.206765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd2b25-5a18-407c-9406-7c17b2178cf2 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:39.213243 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:39.214095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:39.214373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:39.214717 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] PATH_INFO: `/v3/projects/27f8a5a13c3840f587832ae5bc313301` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:39.215831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f8a5a13c3840f587832ae5bc313301)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:39.217721 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:39.220670 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=100 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:39.374805 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f8a5a13c3840f587832ae5bc313301', 'request_id': 'req-c08cf015-34b2-4a93-a9fb-1927f39dce71'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:39.380967 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f8a5a13c3840f587832ae5bc313301', 'request_id': 'req-c08cf015-34b2-4a93-a9fb-1927f39dce71'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:39.387009 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f8a5a13c3840f587832ae5bc313301', 'request_id': 'req-c08cf015-34b2-4a93-a9fb-1927f39dce71'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:39.463604 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] The token cache is being invalidate because project 27f8a5a13c3840f587832ae5bc313301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:12:39.463893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:39.464551 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:39.466178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1426/2849] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:12:39 2022] DELETE /identity/v3/projects/27f8a5a13c3840f587832ae5bc313301 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:39.470105 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83ae58-665f-4636-a31d-cfa5baf7d7b2 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:39.479841 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83ae58-665f-4636-a31d-cfa5baf7d7b2 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:39.535803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:39.536698 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:39.536980 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:39.537255 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] PATH_INFO: `/v3/users/15ed89d88f934365b156f17fbeb22c44` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:39.538431 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ed89d88f934365b156f17fbeb22c44)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:39.540896 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:39.763078 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ed89d88f934365b156f17fbeb22c44'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:39.763745 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ed89d88f934365b156f17fbeb22c44'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:39.775390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ed89d88f934365b156f17fbeb22c44'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:39.778803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ed89d88f934365b156f17fbeb22c44'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:39.782803 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-0d076653-b8bd-4b4b-b4de-348a1cd8aee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ed89d88f934365b156f17fbeb22c44'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:39.847770 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1424/2850] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:12:39 2022] DELETE /identity/v3/users/15ed89d88f934365b156f17fbeb22c44 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:40.289107 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:40.292093 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:40.309821 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cf015-34b2-4a93-a9fb-1927f39dce71 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:40.370468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:40.371325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:40.371624 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:40.371877 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] PATH_INFO: `/v3/projects/205ba6f91b9747e8b8e0bbf791b3034a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:40.373070 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=205ba6f91b9747e8b8e0bbf791b3034a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:40.375007 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:40.519458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '205ba6f91b9747e8b8e0bbf791b3034a', 'request_id': 'req-a7443aa7-c679-4e9d-9b71-bb198ea6e303'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:40.526207 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '205ba6f91b9747e8b8e0bbf791b3034a', 'request_id': 'req-a7443aa7-c679-4e9d-9b71-bb198ea6e303'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:40.534420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '205ba6f91b9747e8b8e0bbf791b3034a', 'request_id': 'req-a7443aa7-c679-4e9d-9b71-bb198ea6e303'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:40.610416 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] The token cache is being invalidate because project 205ba6f91b9747e8b8e0bbf791b3034a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:12:40.610709 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:40.611384 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-a7443aa7-c679-4e9d-9b71-bb198ea6e303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:40.612964 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1427/2851] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:12:40 2022] DELETE /identity/v3/projects/205ba6f91b9747e8b8e0bbf791b3034a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:40.618725 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abfae4-c2d2-4378-8d04-59bad2763e11 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:40.619211 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abfae4-c2d2-4378-8d04-59bad2763e11 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:40.619320 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abfae4-c2d2-4378-8d04-59bad2763e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:40.637702 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-72abfae4-c2d2-4378-8d04-59bad2763e11 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:40.638981 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-72abfae4-c2d2-4378-8d04-59bad2763e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:40.663209 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:40.697990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1425/2852] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:40.705968 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772c416-58a5-4c34-9a36-a895158b753b None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:40.706261 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772c416-58a5-4c34-9a36-a895158b753b None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:40.706547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772c416-58a5-4c34-9a36-a895158b753b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:40.724644 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-e772c416-58a5-4c34-9a36-a895158b753b None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:40.725850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-e772c416-58a5-4c34-9a36-a895158b753b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:40.733929 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1428/2853] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:40.738302 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abfae4-c2d2-4378-8d04-59bad2763e11 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:40.748517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abfae4-c2d2-4378-8d04-59bad2763e11 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:40.753231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355e954-b50d-488a-a3c3-6616c6a3cef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:40.754015 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355e954-b50d-488a-a3c3-6616c6a3cef5 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:40.754625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355e954-b50d-488a-a3c3-6616c6a3cef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:40.754907 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355e954-b50d-488a-a3c3-6616c6a3cef5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:40.755635 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355e954-b50d-488a-a3c3-6616c6a3cef5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:12:40.755947 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355e954-b50d-488a-a3c3-6616c6a3cef5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:40.758119 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355e954-b50d-488a-a3c3-6616c6a3cef5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:40.769758 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1426/2854] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:12:40.774069 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772c416-58a5-4c34-9a36-a895158b753b None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:40.783259 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772c416-58a5-4c34-9a36-a895158b753b None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:40.787894 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c13a7-d5f5-4f9b-a576-a4f15c3af875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:40.788523 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c13a7-d5f5-4f9b-a576-a4f15c3af875 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:40.788811 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c13a7-d5f5-4f9b-a576-a4f15c3af875 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:40.789088 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c13a7-d5f5-4f9b-a576-a4f15c3af875 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:40.789774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c13a7-d5f5-4f9b-a576-a4f15c3af875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:40.791711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c13a7-d5f5-4f9b-a576-a4f15c3af875 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:40.805393 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:40.872373 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1429/2855] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:12:40 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:40.876613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355e954-b50d-488a-a3c3-6616c6a3cef5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:40.887260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355e954-b50d-488a-a3c3-6616c6a3cef5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:40.942878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1781f4e-6def-467b-9c24-8657fc6a474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:40.943563 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1781f4e-6def-467b-9c24-8657fc6a474b admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:40.943843 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1781f4e-6def-467b-9c24-8657fc6a474b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:40.944147 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1781f4e-6def-467b-9c24-8657fc6a474b admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:40.945495 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1781f4e-6def-467b-9c24-8657fc6a474b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:40.947436 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1781f4e-6def-467b-9c24-8657fc6a474b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:41.057492 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1427/2856] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:12:40 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:41.061677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c13a7-d5f5-4f9b-a576-a4f15c3af875 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:41.072335 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c13a7-d5f5-4f9b-a576-a4f15c3af875 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:41.076917 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394f72a-39a0-4546-9129-5478c7b3a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:41.077623 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394f72a-39a0-4546-9129-5478c7b3a41e admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:41.077905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394f72a-39a0-4546-9129-5478c7b3a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:41.078175 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394f72a-39a0-4546-9129-5478c7b3a41e admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:41.078897 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394f72a-39a0-4546-9129-5478c7b3a41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:41.080509 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394f72a-39a0-4546-9129-5478c7b3a41e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:41.090762 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1430/2857] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:41.094759 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1781f4e-6def-467b-9c24-8657fc6a474b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:41.105380 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1781f4e-6def-467b-9c24-8657fc6a474b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:41.110266 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af17df0-09ad-4992-9c38-fe3855348b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:41.110928 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af17df0-09ad-4992-9c38-fe3855348b42 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:41.111214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af17df0-09ad-4992-9c38-fe3855348b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:41.111490 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af17df0-09ad-4992-9c38-fe3855348b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:41.112214 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af17df0-09ad-4992-9c38-fe3855348b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:41.113922 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af17df0-09ad-4992-9c38-fe3855348b42 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:41.124590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1428/2858] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:41.128732 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394f72a-39a0-4546-9129-5478c7b3a41e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:41.138873 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394f72a-39a0-4546-9129-5478c7b3a41e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:41.143628 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3222f4-22d4-4254-a5b2-6af15504a09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:41.144539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3222f4-22d4-4254-a5b2-6af15504a09e admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:41.144832 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3222f4-22d4-4254-a5b2-6af15504a09e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:41.145111 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3222f4-22d4-4254-a5b2-6af15504a09e admin admin] PATH_INFO: `/v3/projects/b30948fef4a042a0a38d08ad90673522/users/e8c62691e15f40eb8259d12bc9938083/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:41.160751 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3222f4-22d4-4254-a5b2-6af15504a09e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30948fef4a042a0a38d08ad90673522, user_id=e8c62691e15f40eb8259d12bc9938083, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:41.163383 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3222f4-22d4-4254-a5b2-6af15504a09e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:41.253209 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1431/2859] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:12:41 2022] PUT /identity/v3/projects/b30948fef4a042a0a38d08ad90673522/users/e8c62691e15f40eb8259d12bc9938083/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:41.264870 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bd504-0df4-4daa-9e91-14504c87a358 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:41.265188 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bd504-0df4-4daa-9e91-14504c87a358 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:41.265502 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bd504-0df4-4daa-9e91-14504c87a358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:41.293470 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-318bd504-0df4-4daa-9e91-14504c87a358 None None] MFA Rules not processed for user `e8c62691e15f40eb8259d12bc9938083`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:41.294668 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-318bd504-0df4-4daa-9e91-14504c87a358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:41.358346 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1429/2860] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:41.369891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3222f4-22d4-4254-a5b2-6af15504a09e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:41.371678 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-bd3222f4-22d4-4254-a5b2-6af15504a09e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:41.428405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3222f4-22d4-4254-a5b2-6af15504a09e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:41.432058 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a22b9-7d2c-497c-9b78-511f80a585b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:41.432852 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a22b9-7d2c-497c-9b78-511f80a585b9 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:41.433132 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a22b9-7d2c-497c-9b78-511f80a585b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:41.433404 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a22b9-7d2c-497c-9b78-511f80a585b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:41.441829 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a22b9-7d2c-497c-9b78-511f80a585b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:41.443313 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a22b9-7d2c-497c-9b78-511f80a585b9 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:41.452425 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1432/2861] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:41.471552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bd504-0df4-4daa-9e91-14504c87a358 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:41.500515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:41.537305 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bd504-0df4-4daa-9e91-14504c87a358 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:41.542570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:41.543520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:41.543807 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:41.544097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] PATH_INFO: `/v3/users/e8c62691e15f40eb8259d12bc9938083` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:41.545343 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c62691e15f40eb8259d12bc9938083)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:41.547247 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:41.770638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c62691e15f40eb8259d12bc9938083'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:41.771347 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c62691e15f40eb8259d12bc9938083'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:41.784628 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c62691e15f40eb8259d12bc9938083'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:41.785273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c62691e15f40eb8259d12bc9938083'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:41.792058 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c62691e15f40eb8259d12bc9938083'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:41.864763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1430/2862] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:12:41 2022] DELETE /identity/v3/users/e8c62691e15f40eb8259d12bc9938083 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:41.869197 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a710f-67a0-4ea7-a814-484a9a03f1a4 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:41.869492 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a710f-67a0-4ea7-a814-484a9a03f1a4 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:41.869775 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a710f-67a0-4ea7-a814-484a9a03f1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:41.887468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-676a710f-67a0-4ea7-a814-484a9a03f1a4 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:41.888711 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-676a710f-67a0-4ea7-a814-484a9a03f1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:41.947176 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1433/2863] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:41.961297 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:41.962782 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:42.020448 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab040926-2250-4bcd-829d-da63a5922059 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:42.024764 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2d6ca-0c45-45e9-85a0-a97e7e84c3c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:42.025356 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2d6ca-0c45-45e9-85a0-a97e7e84c3c9 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:42.025641 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2d6ca-0c45-45e9-85a0-a97e7e84c3c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:42.025916 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2d6ca-0c45-45e9-85a0-a97e7e84c3c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:42.035070 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2d6ca-0c45-45e9-85a0-a97e7e84c3c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:42.036542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2d6ca-0c45-45e9-85a0-a97e7e84c3c9 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:42.046018 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1431/2864] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:42.449353 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a710f-67a0-4ea7-a814-484a9a03f1a4 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:42.450947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-676a710f-67a0-4ea7-a814-484a9a03f1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:42.465816 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a710f-67a0-4ea7-a814-484a9a03f1a4 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:42.470467 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:42.471382 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:42.471598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:42.471961 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] PATH_INFO: `/v3/projects/b30948fef4a042a0a38d08ad90673522` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:42.473113 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30948fef4a042a0a38d08ad90673522)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:42.475008 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:42.578598 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30948fef4a042a0a38d08ad90673522', 'request_id': 'req-8650e73d-e339-493b-ab27-4f5aee045550'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:42.584573 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30948fef4a042a0a38d08ad90673522', 'request_id': 'req-8650e73d-e339-493b-ab27-4f5aee045550'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:42.586696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:42.590447 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30948fef4a042a0a38d08ad90673522', 'request_id': 'req-8650e73d-e339-493b-ab27-4f5aee045550'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:42.675355 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] The token cache is being invalidate because project b30948fef4a042a0a38d08ad90673522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:12:42.675748 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:42.676431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-8650e73d-e339-493b-ab27-4f5aee045550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:12:42.678012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1434/2865] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:12:42 2022] DELETE /identity/v3/projects/b30948fef4a042a0a38d08ad90673522 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:42.683602 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2705e10-b723-4a04-840a-299fc8562d51 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:42.684079 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2705e10-b723-4a04-840a-299fc8562d51 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:42.684222 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2705e10-b723-4a04-840a-299fc8562d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:42.701828 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-e2705e10-b723-4a04-840a-299fc8562d51 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:42.703061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-e2705e10-b723-4a04-840a-299fc8562d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:42.751476 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:42.769317 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1432/2866] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:42.777539 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c0ecc-590b-45f1-9951-3b287eade2ec None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:42.777833 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c0ecc-590b-45f1-9951-3b287eade2ec None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:42.778120 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c0ecc-590b-45f1-9951-3b287eade2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:42.797942 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-117c0ecc-590b-45f1-9951-3b287eade2ec None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:42.799167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-117c0ecc-590b-45f1-9951-3b287eade2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:42.807690 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1435/2867] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:42.811991 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2705e10-b723-4a04-840a-299fc8562d51 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:42.821763 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2705e10-b723-4a04-840a-299fc8562d51 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:42.826310 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469c3b9-6b86-4897-a838-cb14823748e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:42.826970 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469c3b9-6b86-4897-a838-cb14823748e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:42.827251 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469c3b9-6b86-4897-a838-cb14823748e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:42.827528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469c3b9-6b86-4897-a838-cb14823748e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:42.828276 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469c3b9-6b86-4897-a838-cb14823748e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:12:42.828663 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469c3b9-6b86-4897-a838-cb14823748e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:42.830196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469c3b9-6b86-4897-a838-cb14823748e4 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:42.842630 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1433/2868] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:12:42.847031 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c0ecc-590b-45f1-9951-3b287eade2ec None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:42.857321 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c0ecc-590b-45f1-9951-3b287eade2ec None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:42.862577 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e38a76-3874-44d9-a63c-47102893adb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:42.863227 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e38a76-3874-44d9-a63c-47102893adb8 admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:42.863510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e38a76-3874-44d9-a63c-47102893adb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:42.863783 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e38a76-3874-44d9-a63c-47102893adb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:42.864514 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e38a76-3874-44d9-a63c-47102893adb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:42.866405 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e38a76-3874-44d9-a63c-47102893adb8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:42.965647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1436/2869] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:12:42 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:42.969859 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469c3b9-6b86-4897-a838-cb14823748e4 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:42.980497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469c3b9-6b86-4897-a838-cb14823748e4 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:43.039548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f342298-ae2b-4b7a-a3e3-f75870c627b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:43.040229 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f342298-ae2b-4b7a-a3e3-f75870c627b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:43.040548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f342298-ae2b-4b7a-a3e3-f75870c627b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:43.040823 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f342298-ae2b-4b7a-a3e3-f75870c627b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:43.041537 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f342298-ae2b-4b7a-a3e3-f75870c627b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:43.043483 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f342298-ae2b-4b7a-a3e3-f75870c627b7 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:43.125528 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1434/2870] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:12:42 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:12:43.129562 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e38a76-3874-44d9-a63c-47102893adb8 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:43.139919 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e38a76-3874-44d9-a63c-47102893adb8 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:43.144697 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22987c82-63f4-467b-beeb-661be5028ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:43.145354 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22987c82-63f4-467b-beeb-661be5028ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:43.145639 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22987c82-63f4-467b-beeb-661be5028ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:43.145970 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22987c82-63f4-467b-beeb-661be5028ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:43.146704 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22987c82-63f4-467b-beeb-661be5028ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:43.148887 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22987c82-63f4-467b-beeb-661be5028ff8 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:43.159279 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1437/2871] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:43.163372 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f342298-ae2b-4b7a-a3e3-f75870c627b7 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:43.173979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f342298-ae2b-4b7a-a3e3-f75870c627b7 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:43.178590 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64330c3a-0517-4262-8a6e-461c4593d7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:43.179253 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64330c3a-0517-4262-8a6e-461c4593d7cf admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:43.179538 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64330c3a-0517-4262-8a6e-461c4593d7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:43.179815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64330c3a-0517-4262-8a6e-461c4593d7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:43.180571 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64330c3a-0517-4262-8a6e-461c4593d7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:43.182148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64330c3a-0517-4262-8a6e-461c4593d7cf admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:43.192059 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1435/2872] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:12:43.196196 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22987c82-63f4-467b-beeb-661be5028ff8 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:43.206170 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22987c82-63f4-467b-beeb-661be5028ff8 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:43.211309 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648953a5-a49b-415d-adfe-f35024e4df23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:43.212184 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648953a5-a49b-415d-adfe-f35024e4df23 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:43.212506 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648953a5-a49b-415d-adfe-f35024e4df23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:43.212791 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648953a5-a49b-415d-adfe-f35024e4df23 admin admin] PATH_INFO: `/v3/projects/3ac1dac1f89649e0929a000f6d6b8965/users/5ddd0a47f2764f3da778bdda04701a14/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:43.228005 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648953a5-a49b-415d-adfe-f35024e4df23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ac1dac1f89649e0929a000f6d6b8965, user_id=5ddd0a47f2764f3da778bdda04701a14, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:43.230687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648953a5-a49b-415d-adfe-f35024e4df23 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:43.348431 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1438/2873] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:12:43 2022] PUT /identity/v3/projects/3ac1dac1f89649e0929a000f6d6b8965/users/5ddd0a47f2764f3da778bdda04701a14/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:12:43.359144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a00e07-ff6a-4e0f-8673-a3d2b5c3fe23 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:43.359437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a00e07-ff6a-4e0f-8673-a3d2b5c3fe23 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:43.359723 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a00e07-ff6a-4e0f-8673-a3d2b5c3fe23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:43.391309 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-46a00e07-ff6a-4e0f-8673-a3d2b5c3fe23 None None] MFA Rules not processed for user `5ddd0a47f2764f3da778bdda04701a14`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:12:43.392515 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-46a00e07-ff6a-4e0f-8673-a3d2b5c3fe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:43.461935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1436/2874] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:12:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:12:43.473200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648953a5-a49b-415d-adfe-f35024e4df23 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:12:43.475022 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-648953a5-a49b-415d-adfe-f35024e4df23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:12:43.536232 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648953a5-a49b-415d-adfe-f35024e4df23 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:12:43.540012 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b479756-04a2-4056-8bf3-0f987d15ed86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d30c8d1f883400ba397ea09d42a0aa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:12:43.540725 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b479756-04a2-4056-8bf3-0f987d15ed86 service cinder] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:12:43.541013 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b479756-04a2-4056-8bf3-0f987d15ed86 service cinder] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:12:43.541294 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b479756-04a2-4056-8bf3-0f987d15ed86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:12:43.550643 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b479756-04a2-4056-8bf3-0f987d15ed86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:12:43.552113 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b479756-04a2-4056-8bf3-0f987d15ed86 service cinder] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:12:43.562116 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1439/2875] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:12:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:12:53.458196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:12:53.557640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=57,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:03.754775 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a00e07-ff6a-4e0f-8673-a3d2b5c3fe23 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:03.817722 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a00e07-ff6a-4e0f-8673-a3d2b5c3fe23 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:03.822415 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:03.823336 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:03.823618 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:03.823894 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] PATH_INFO: `/v3/users/5ddd0a47f2764f3da778bdda04701a14` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:03.825228 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddd0a47f2764f3da778bdda04701a14)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:03.827203 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:04.022056 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddd0a47f2764f3da778bdda04701a14'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:13:04.022706 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddd0a47f2764f3da778bdda04701a14'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:13:04.036518 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddd0a47f2764f3da778bdda04701a14'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:13:04.037271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddd0a47f2764f3da778bdda04701a14'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:13:04.046925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddd0a47f2764f3da778bdda04701a14'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:13:04.105428 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1437/2876] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:13:03 2022] DELETE /identity/v3/users/5ddd0a47f2764f3da778bdda04701a14 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:13:04.112468 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb5a0b-4c24-4e0c-a23a-b43ccf5c70ff None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:04.113064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb5a0b-4c24-4e0c-a23a-b43ccf5c70ff None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:04.113064 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb5a0b-4c24-4e0c-a23a-b43ccf5c70ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:04.132021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-7ceb5a0b-4c24-4e0c-a23a-b43ccf5c70ff None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:13:04.133284 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7ceb5a0b-4c24-4e0c-a23a-b43ccf5c70ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:04.194174 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1440/2877] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:13:04.204652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:04.206093 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:04.264979 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9b561-16da-4917-ba1a-0eb04edb2b3b admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:04.268754 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644079d-24fa-4530-8330-9d8325978b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:04.269406 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644079d-24fa-4530-8330-9d8325978b55 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:04.269687 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644079d-24fa-4530-8330-9d8325978b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:04.269965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644079d-24fa-4530-8330-9d8325978b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:04.280605 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644079d-24fa-4530-8330-9d8325978b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:04.282099 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644079d-24fa-4530-8330-9d8325978b55 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:04.296935 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1438/2878] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:13:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:13:04.701420 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb5a0b-4c24-4e0c-a23a-b43ccf5c70ff None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:04.702889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-7ceb5a0b-4c24-4e0c-a23a-b43ccf5c70ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:04.715205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb5a0b-4c24-4e0c-a23a-b43ccf5c70ff None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:04.720276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:04.721696 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:04.721992 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:04.722281 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] PATH_INFO: `/v3/projects/3ac1dac1f89649e0929a000f6d6b8965` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:04.723517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ac1dac1f89649e0929a000f6d6b8965)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:04.725521 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:04.835621 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ac1dac1f89649e0929a000f6d6b8965', 'request_id': 'req-066cc1de-915c-4676-a50d-78b0ff3b0503'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:13:04.841994 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ac1dac1f89649e0929a000f6d6b8965', 'request_id': 'req-066cc1de-915c-4676-a50d-78b0ff3b0503'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:13:04.847613 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ac1dac1f89649e0929a000f6d6b8965', 'request_id': 'req-066cc1de-915c-4676-a50d-78b0ff3b0503'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:13:04.913510 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] The token cache is being invalidate because project 3ac1dac1f89649e0929a000f6d6b8965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:13:04.913803 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:13:04.914456 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-066cc1de-915c-4676-a50d-78b0ff3b0503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:13:04.916036 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1441/2879] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:13:04 2022] DELETE /identity/v3/projects/3ac1dac1f89649e0929a000f6d6b8965 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:13:04.922019 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d76ae-69c4-4d18-9fe9-89ebe7f06097 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:04.922616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d76ae-69c4-4d18-9fe9-89ebe7f06097 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:04.922616 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d76ae-69c4-4d18-9fe9-89ebe7f06097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:04.942350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-107d76ae-69c4-4d18-9fe9-89ebe7f06097 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:13:04.947233 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-107d76ae-69c4-4d18-9fe9-89ebe7f06097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:04.960802 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:05.006566 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1439/2880] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:13:05.014712 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a3d44-7cd7-411d-a164-ca30c3df8962 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:05.015001 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a3d44-7cd7-411d-a164-ca30c3df8962 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:05.015445 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a3d44-7cd7-411d-a164-ca30c3df8962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:05.035081 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-c08a3d44-7cd7-411d-a164-ca30c3df8962 None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:13:05.036301 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c08a3d44-7cd7-411d-a164-ca30c3df8962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:05.044899 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1442/2881] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:13:05.049353 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d76ae-69c4-4d18-9fe9-89ebe7f06097 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:05.061792 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d76ae-69c4-4d18-9fe9-89ebe7f06097 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:05.073790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb76e-f887-4141-a147-c0e80c1c20a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:05.073790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb76e-f887-4141-a147-c0e80c1c20a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:05.073790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb76e-f887-4141-a147-c0e80c1c20a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:05.073790 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb76e-f887-4141-a147-c0e80c1c20a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:05.074231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeb76e-f887-4141-a147-c0e80c1c20a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87074) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:13:05.074231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeb76e-f887-4141-a147-c0e80c1c20a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:05.074231 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeb76e-f887-4141-a147-c0e80c1c20a5 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:05.084039 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1440/2882] 10.222.0.23 () {58 vars in 1152 bytes} [Mon May 23 16:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:13:05.088677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a3d44-7cd7-411d-a164-ca30c3df8962 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:05.098027 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a3d44-7cd7-411d-a164-ca30c3df8962 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:05.104273 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8cf2-839a-469b-8409-281e6a8322ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:05.104963 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8cf2-839a-469b-8409-281e6a8322ad admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:05.105245 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8cf2-839a-469b-8409-281e6a8322ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:05.105517 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8cf2-839a-469b-8409-281e6a8322ad admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:05.106263 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a8cf2-839a-469b-8409-281e6a8322ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:05.109000 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a8cf2-839a-469b-8409-281e6a8322ad admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:05.209466 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1443/2883] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:13:05 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:13:05.214485 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb76e-f887-4141-a147-c0e80c1c20a5 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:05.224960 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb76e-f887-4141-a147-c0e80c1c20a5 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:05.285077 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535aca98-dd59-4b95-b1cb-c464ef94ea74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:05.286195 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aca98-dd59-4b95-b1cb-c464ef94ea74 admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:05.286478 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aca98-dd59-4b95-b1cb-c464ef94ea74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:05.286759 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aca98-dd59-4b95-b1cb-c464ef94ea74 admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:05.287464 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535aca98-dd59-4b95-b1cb-c464ef94ea74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:05.289432 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535aca98-dd59-4b95-b1cb-c464ef94ea74 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:05.405280 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1441/2884] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:13:05 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:13:05.409479 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8cf2-839a-469b-8409-281e6a8322ad admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:05.420618 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8cf2-839a-469b-8409-281e6a8322ad admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:05.425295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd61d12-67a5-4ec2-9945-dc9ce78b25e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:05.429030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61d12-67a5-4ec2-9945-dc9ce78b25e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:05.429030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61d12-67a5-4ec2-9945-dc9ce78b25e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:05.429030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61d12-67a5-4ec2-9945-dc9ce78b25e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:05.429030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd61d12-67a5-4ec2-9945-dc9ce78b25e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:05.429030 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd61d12-67a5-4ec2-9945-dc9ce78b25e0 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:05.439626 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1444/2885] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:13:05.443871 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535aca98-dd59-4b95-b1cb-c464ef94ea74 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:05.454271 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535aca98-dd59-4b95-b1cb-c464ef94ea74 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:05.459098 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871f26a-5fff-4f18-8751-a917ccbe4ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:05.459760 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871f26a-5fff-4f18-8751-a917ccbe4ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:05.460040 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871f26a-5fff-4f18-8751-a917ccbe4ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:05.460312 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871f26a-5fff-4f18-8751-a917ccbe4ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:05.461097 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871f26a-5fff-4f18-8751-a917ccbe4ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:05.462676 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871f26a-5fff-4f18-8751-a917ccbe4ae0 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:05.475260 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1442/2886] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:13:05.484556 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd61d12-67a5-4ec2-9945-dc9ce78b25e0 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:05.494758 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd61d12-67a5-4ec2-9945-dc9ce78b25e0 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:05.499504 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d12d7-ff87-4b8f-8b19-73ab5d80cd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:05.500408 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d12d7-ff87-4b8f-8b19-73ab5d80cd14 admin admin] REQUEST_METHOD: `PUT` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:05.500687 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d12d7-ff87-4b8f-8b19-73ab5d80cd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:05.500957 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d12d7-ff87-4b8f-8b19-73ab5d80cd14 admin admin] PATH_INFO: `/v3/projects/3fe59cde61bd452592e3d52217b1329a/users/570744ca9e494bf9901c1552f7a29e7c/roles/177939861d6e4e82b48401be184d66b4` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:05.515701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d12d7-ff87-4b8f-8b19-73ab5d80cd14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe59cde61bd452592e3d52217b1329a, user_id=570744ca9e494bf9901c1552f7a29e7c, role_id=177939861d6e4e82b48401be184d66b4)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:05.518357 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d12d7-ff87-4b8f-8b19-73ab5d80cd14 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:05.590606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1445/2887] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:13:05 2022] PUT /identity/v3/projects/3fe59cde61bd452592e3d52217b1329a/users/570744ca9e494bf9901c1552f7a29e7c/roles/177939861d6e4e82b48401be184d66b4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:13:05.596201 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37e9fe-5fd1-4574-aa93-f05754e6e7df None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:05.596522 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37e9fe-5fd1-4574-aa93-f05754e6e7df None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:05.596915 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37e9fe-5fd1-4574-aa93-f05754e6e7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:05.618517 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-cc37e9fe-5fd1-4574-aa93-f05754e6e7df None None] MFA Rules not processed for user `0840c2e542b841edbc74432280137f72`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:13:05.619750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-cc37e9fe-5fd1-4574-aa93-f05754e6e7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:05.684451 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1443/2888] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:13:05.694216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d12d7-ff87-4b8f-8b19-73ab5d80cd14 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:05.695981 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-cc2d12d7-ff87-4b8f-8b19-73ab5d80cd14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:05.711246 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:05.756438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d12d7-ff87-4b8f-8b19-73ab5d80cd14 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:05.760141 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85756f0d-4467-4305-8656-ecf4fa54df25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:05.760851 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756f0d-4467-4305-8656-ecf4fa54df25 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:05.761438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756f0d-4467-4305-8656-ecf4fa54df25 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:05.761893 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85756f0d-4467-4305-8656-ecf4fa54df25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:05.774200 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85756f0d-4467-4305-8656-ecf4fa54df25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:05.775693 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85756f0d-4467-4305-8656-ecf4fa54df25 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:05.790097 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1446/2889] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:13:11.168067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd7015-6e59-41be-99f5-9cd0f4ef1775 None None] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:11.168625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd7015-6e59-41be-99f5-9cd0f4ef1775 None None] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:11.168625 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd7015-6e59-41be-99f5-9cd0f4ef1775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:11.198721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.auth.core [None req-81bd7015-6e59-41be-99f5-9cd0f4ef1775 None None] MFA Rules not processed for user `570744ca9e494bf9901c1552f7a29e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=87074) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:13:11.200146 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-81bd7015-6e59-41be-99f5-9cd0f4ef1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:11.218390 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:11.269827 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1444/2890] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:13:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:13:11.274412 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85756f0d-4467-4305-8656-ecf4fa54df25 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:11.289477 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85756f0d-4467-4305-8656-ecf4fa54df25 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:11.295164 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e3ef5-73d5-4bdd-8106-f8b27b4bbb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:11.295835 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e3ef5-73d5-4bdd-8106-f8b27b4bbb6e admin admin] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:11.296121 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e3ef5-73d5-4bdd-8106-f8b27b4bbb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:11.296426 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e3ef5-73d5-4bdd-8106-f8b27b4bbb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:11.297126 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e3ef5-73d5-4bdd-8106-f8b27b4bbb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:11.299049 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e3ef5-73d5-4bdd-8106-f8b27b4bbb6e admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:11.386570 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1447/2891] 10.222.0.23 () {60 vars in 1152 bytes} [Mon May 23 16:13:11 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:13:11.390714 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd7015-6e59-41be-99f5-9cd0f4ef1775 None None] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:11.402032 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd7015-6e59-41be-99f5-9cd0f4ef1775 None None] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:11.467834 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11513122-b918-4463-acc6-a04840be42ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:11.468570 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11513122-b918-4463-acc6-a04840be42ee admin admin] REQUEST_METHOD: `POST` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:11.468858 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11513122-b918-4463-acc6-a04840be42ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:11.469133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11513122-b918-4463-acc6-a04840be42ee admin admin] PATH_INFO: `/v3/users` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:11.469844 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11513122-b918-4463-acc6-a04840be42ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:11.471780 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11513122-b918-4463-acc6-a04840be42ee admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:11.583179 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1445/2892] 10.222.0.23 () {60 vars in 1143 bytes} [Mon May 23 16:13:11 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:13:11.587086 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e3ef5-73d5-4bdd-8106-f8b27b4bbb6e admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:11.597891 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e3ef5-73d5-4bdd-8106-f8b27b4bbb6e admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:11.602473 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9829754-262d-4ffd-b68a-c4559ed092f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:11.603130 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9829754-262d-4ffd-b68a-c4559ed092f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:11.603419 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9829754-262d-4ffd-b68a-c4559ed092f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:11.603700 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9829754-262d-4ffd-b68a-c4559ed092f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:11.604458 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9829754-262d-4ffd-b68a-c4559ed092f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:11.606053 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9829754-262d-4ffd-b68a-c4559ed092f3 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:11.617328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1448/2893] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:13:11.623422 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11513122-b918-4463-acc6-a04840be42ee admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:11.633600 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11513122-b918-4463-acc6-a04840be42ee admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:11.638652 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a65ed-2da8-4829-9409-325badf61fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:11.639536 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a65ed-2da8-4829-9409-325badf61fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:11.639814 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a65ed-2da8-4829-9409-325badf61fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:11.640095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a65ed-2da8-4829-9409-325badf61fbf admin admin] PATH_INFO: `/v3/projects/b665541eff14427895aa4c5faa87d822/users/09b408b4a93c48d69824adf9122b183e/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:11.656540 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a65ed-2da8-4829-9409-325badf61fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b665541eff14427895aa4c5faa87d822, user_id=09b408b4a93c48d69824adf9122b183e, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:11.659150 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a65ed-2da8-4829-9409-325badf61fbf admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:11.725151 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1446/2894] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:13:11 2022] PUT /identity/v3/projects/b665541eff14427895aa4c5faa87d822/users/09b408b4a93c48d69824adf9122b183e/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:13:11.729205 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9829754-262d-4ffd-b68a-c4559ed092f3 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:11.739580 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9829754-262d-4ffd-b68a-c4559ed092f3 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:11.798966 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313b581-4d9d-4fc4-ad6d-222beb5c1954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:11.799640 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313b581-4d9d-4fc4-ad6d-222beb5c1954 admin admin] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:11.799978 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313b581-4d9d-4fc4-ad6d-222beb5c1954 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:11.800262 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313b581-4d9d-4fc4-ad6d-222beb5c1954 admin admin] PATH_INFO: `/v3/roles` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:11.801021 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313b581-4d9d-4fc4-ad6d-222beb5c1954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:11.802614 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313b581-4d9d-4fc4-ad6d-222beb5c1954 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:11.812167 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1449/2895] 10.222.0.23 () {58 vars in 1121 bytes} [Mon May 23 16:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:13:11.816274 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a65ed-2da8-4829-9409-325badf61fbf admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:11.826966 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a65ed-2da8-4829-9409-325badf61fbf admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:11.831497 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55955a46-781f-4802-9e64-cfd41a09c4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:11.832384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55955a46-781f-4802-9e64-cfd41a09c4e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:11.832682 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55955a46-781f-4802-9e64-cfd41a09c4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:11.832957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55955a46-781f-4802-9e64-cfd41a09c4e1 admin admin] PATH_INFO: `/v3/projects/b665541eff14427895aa4c5faa87d822/users/09b408b4a93c48d69824adf9122b183e/roles/04c11dea81274a69974ad5c639f2c60b` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:11.834910 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55955a46-781f-4802-9e64-cfd41a09c4e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b665541eff14427895aa4c5faa87d822, user_id=09b408b4a93c48d69824adf9122b183e, role_id=04c11dea81274a69974ad5c639f2c60b)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:11.837552 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55955a46-781f-4802-9e64-cfd41a09c4e1 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:11.848239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1447/2896] 10.222.0.23 () {60 vars in 1482 bytes} [Mon May 23 16:13:11 2022] PUT /identity/v3/projects/b665541eff14427895aa4c5faa87d822/users/09b408b4a93c48d69824adf9122b183e/roles/04c11dea81274a69974ad5c639f2c60b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:13:17.394915 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9752e0-90c9-4ed0-8de4-cc7ac0d04a67 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:17.395478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9752e0-90c9-4ed0-8de4-cc7ac0d04a67 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:17.395478 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9752e0-90c9-4ed0-8de4-cc7ac0d04a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:17.452399 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-1f9752e0-90c9-4ed0-8de4-cc7ac0d04a67 None None] MFA Rules not processed for user `09b408b4a93c48d69824adf9122b183e`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:13:17.453661 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1f9752e0-90c9-4ed0-8de4-cc7ac0d04a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:17.531302 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1450/2897] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:13:17 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:13:17.547458 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55955a46-781f-4802-9e64-cfd41a09c4e1 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:17.549359 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-55955a46-781f-4802-9e64-cfd41a09c4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:17.628516 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55955a46-781f-4802-9e64-cfd41a09c4e1 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:17.634102 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb47892-f126-4934-a580-44873cedb9d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:17.634774 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb47892-f126-4934-a580-44873cedb9d9 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:17.635067 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb47892-f126-4934-a580-44873cedb9d9 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:17.635350 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb47892-f126-4934-a580-44873cedb9d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:17.642138 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:17.697887 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb47892-f126-4934-a580-44873cedb9d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:17.699683 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb47892-f126-4934-a580-44873cedb9d9 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:17.709866 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1448/2898] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:13:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:13:20.196181 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdc792-9819-47b1-80e2-f4d96312c334 None None] REQUEST_METHOD: `POST` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:20.196656 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdc792-9819-47b1-80e2-f4d96312c334 None None] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:20.196781 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdc792-9819-47b1-80e2-f4d96312c334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:20.205705 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:20.219297 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.auth.core [None req-4cfdc792-9819-47b1-80e2-f4d96312c334 None None] MFA Rules not processed for user `570744ca9e494bf9901c1552f7a29e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=87073) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:13:20.220460 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4cfdc792-9819-47b1-80e2-f4d96312c334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:20.227847 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1451/2899] 10.222.0.23 () {58 vars in 957 bytes} [Mon May 23 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:13:27.759256 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=24 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:30.508178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=2 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:36.403551 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb47892-f126-4934-a580-44873cedb9d9 service nova] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:36.405133 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-5cb47892-f126-4934-a580-44873cedb9d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:36.468239 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb47892-f126-4934-a580-44873cedb9d9 service nova] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:36.471864 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c84196-e057-406b-bef8-6989a2e9b601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:36.472542 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c84196-e057-406b-bef8-6989a2e9b601 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:36.472842 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c84196-e057-406b-bef8-6989a2e9b601 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:36.473169 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c84196-e057-406b-bef8-6989a2e9b601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:36.474925 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-03c84196-e057-406b-bef8-6989a2e9b601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:36.485319 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c84196-e057-406b-bef8-6989a2e9b601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:36.486721 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c84196-e057-406b-bef8-6989a2e9b601 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:36.494728 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1449/2900] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:13:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:13:38.187163 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfdc792-9819-47b1-80e2-f4d96312c334 None None] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:38.188581 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-4cfdc792-9819-47b1-80e2-f4d96312c334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:38.259692 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfdc792-9819-47b1-80e2-f4d96312c334 None None] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:38.265601 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c23317-1fc3-442c-8fc2-2efa3ee8436c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:38.266260 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c23317-1fc3-442c-8fc2-2efa3ee8436c service placement] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:38.266547 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c23317-1fc3-442c-8fc2-2efa3ee8436c service placement] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:38.266827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c23317-1fc3-442c-8fc2-2efa3ee8436c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:38.268647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-48c23317-1fc3-442c-8fc2-2efa3ee8436c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:38.280819 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c23317-1fc3-442c-8fc2-2efa3ee8436c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:38.282422 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c23317-1fc3-442c-8fc2-2efa3ee8436c service placement] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:38.294358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1452/2901] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:13:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:13:44.359419 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c84196-e057-406b-bef8-6989a2e9b601 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:44.368614 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c84196-e057-406b-bef8-6989a2e9b601 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:44.372927 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125809de-bbd3-49e7-81d9-f6f29fe88285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:44.377411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125809de-bbd3-49e7-81d9-f6f29fe88285 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:44.377411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125809de-bbd3-49e7-81d9-f6f29fe88285 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:44.377411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125809de-bbd3-49e7-81d9-f6f29fe88285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:44.377411 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-125809de-bbd3-49e7-81d9-f6f29fe88285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:44.387051 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125809de-bbd3-49e7-81d9-f6f29fe88285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:44.388990 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125809de-bbd3-49e7-81d9-f6f29fe88285 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:44.398957 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1450/2902] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:13:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:13:48.288677 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=32 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:54.395030 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=32 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:13:55.877765 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c23317-1fc3-442c-8fc2-2efa3ee8436c service placement] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:55.888553 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c23317-1fc3-442c-8fc2-2efa3ee8436c service placement] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:55.894282 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e19ea-16dc-4b0c-ba71-a6483e8a9071 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:55.894930 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e19ea-16dc-4b0c-ba71-a6483e8a9071 service nova] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:55.895216 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e19ea-16dc-4b0c-ba71-a6483e8a9071 service nova] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:55.895526 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e19ea-16dc-4b0c-ba71-a6483e8a9071 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:55.897579 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-228e19ea-16dc-4b0c-ba71-a6483e8a9071 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:13:55.910099 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e19ea-16dc-4b0c-ba71-a6483e8a9071 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:55.911682 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e19ea-16dc-4b0c-ba71-a6483e8a9071 service nova] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:55.922328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1453/2903] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:13:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:13:58.078170 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125809de-bbd3-49e7-81d9-f6f29fe88285 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:13:58.087106 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125809de-bbd3-49e7-81d9-f6f29fe88285 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:13:58.090696 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5025943-3ff5-4207-94fa-3e8401a95847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:13:58.091831 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5025943-3ff5-4207-94fa-3e8401a95847 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:13:58.092112 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5025943-3ff5-4207-94fa-3e8401a95847 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:13:58.092408 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5025943-3ff5-4207-94fa-3e8401a95847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:13:58.101273 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5025943-3ff5-4207-94fa-3e8401a95847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:13:58.102805 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5025943-3ff5-4207-94fa-3e8401a95847 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:13:58.112951 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1451/2904] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:13:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:14:05.917831 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=6 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:08.108808 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=6 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:14:20.204325 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e19ea-16dc-4b0c-ba71-a6483e8a9071 service nova] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:14:20.216342 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e19ea-16dc-4b0c-ba71-a6483e8a9071 service nova] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:14:20.222252 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a29d2-bd65-4f66-95b1-8cd0951186fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:14:20.222907 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a29d2-bd65-4f66-95b1-8cd0951186fb service placement] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:14:20.223256 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a29d2-bd65-4f66-95b1-8cd0951186fb service placement] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:14:20.223647 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a29d2-bd65-4f66-95b1-8cd0951186fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:14:20.225524 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-048a29d2-bd65-4f66-95b1-8cd0951186fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:14:20.238443 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a29d2-bd65-4f66-95b1-8cd0951186fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:14:20.240011 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a29d2-bd65-4f66-95b1-8cd0951186fb service placement] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:14:20.249905 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1454/2905] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:14:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:14:30.245638 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=6 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:09.847865 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5025943-3ff5-4207-94fa-3e8401a95847 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:16:09.857263 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5025943-3ff5-4207-94fa-3e8401a95847 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:16:09.861608 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585c067-1b65-4ac7-8a60-63dbfe987aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:16:09.862242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585c067-1b65-4ac7-8a60-63dbfe987aca service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:16:09.862520 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585c067-1b65-4ac7-8a60-63dbfe987aca service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:16:09.862795 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585c067-1b65-4ac7-8a60-63dbfe987aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:16:09.864872 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-2585c067-1b65-4ac7-8a60-63dbfe987aca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:16:09.878868 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585c067-1b65-4ac7-8a60-63dbfe987aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:16:09.880692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585c067-1b65-4ac7-8a60-63dbfe987aca service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:16:09.892940 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1452/2906] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:16:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:16:11.402767 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a29d2-bd65-4f66-95b1-8cd0951186fb service placement] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:16:11.412470 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a29d2-bd65-4f66-95b1-8cd0951186fb service placement] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:16:11.416114 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3486236-8fe9-48a7-90f7-66e6ef39abac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:16:11.416884 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3486236-8fe9-48a7-90f7-66e6ef39abac service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:16:11.417173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3486236-8fe9-48a7-90f7-66e6ef39abac service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:16:11.417453 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3486236-8fe9-48a7-90f7-66e6ef39abac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:16:11.419328 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-b3486236-8fe9-48a7-90f7-66e6ef39abac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:16:11.429858 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3486236-8fe9-48a7-90f7-66e6ef39abac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:16:11.431295 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3486236-8fe9-48a7-90f7-66e6ef39abac service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:16:11.440606 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1455/2907] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:16:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:16:11.521113 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585c067-1b65-4ac7-8a60-63dbfe987aca service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:16:11.529953 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585c067-1b65-4ac7-8a60-63dbfe987aca service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:16:11.533766 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6251f92-9f1f-427c-a8fc-ba9f6a3abbe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:16:11.537369 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251f92-9f1f-427c-a8fc-ba9f6a3abbe9 service neutron] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:16:11.537369 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251f92-9f1f-427c-a8fc-ba9f6a3abbe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:16:11.537369 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6251f92-9f1f-427c-a8fc-ba9f6a3abbe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:16:11.537724 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-b6251f92-9f1f-427c-a8fc-ba9f6a3abbe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:16:11.552474 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6251f92-9f1f-427c-a8fc-ba9f6a3abbe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:16:11.553920 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6251f92-9f1f-427c-a8fc-ba9f6a3abbe9 service neutron] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:16:11.565410 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1453/2908] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:16:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:16:12.530290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3486236-8fe9-48a7-90f7-66e6ef39abac service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:16:12.539029 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3486236-8fe9-48a7-90f7-66e6ef39abac service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:16:12.542760 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882888bf-5267-48df-9420-8ec6f1b9ade6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39f5189d63d467dbd914f553c261c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:16:12.543318 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882888bf-5267-48df-9420-8ec6f1b9ade6 service neutron] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:16:12.543603 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882888bf-5267-48df-9420-8ec6f1b9ade6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:16:12.543880 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882888bf-5267-48df-9420-8ec6f1b9ade6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:16:12.545667 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-882888bf-5267-48df-9420-8ec6f1b9ade6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:16:12.558850 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882888bf-5267-48df-9420-8ec6f1b9ade6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:16:12.560265 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882888bf-5267-48df-9420-8ec6f1b9ade6 service neutron] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:16:12.571540 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1456/2909] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:16:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:16:21.560096 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=12 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:16:22.566827 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=12 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:40.163186 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6251f92-9f1f-427c-a8fc-ba9f6a3abbe9 service neutron] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:17:40.175569 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6251f92-9f1f-427c-a8fc-ba9f6a3abbe9 service neutron] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:17:40.181373 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030004d-0a32-42fc-9f7d-289386c0ffa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9129f0b362a4945a0eca7cc33571abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:17:40.182047 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030004d-0a32-42fc-9f7d-289386c0ffa4 service nova] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:17:40.182337 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030004d-0a32-42fc-9f7d-289386c0ffa4 service nova] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:17:40.182620 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030004d-0a32-42fc-9f7d-289386c0ffa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:17:40.184548 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-a030004d-0a32-42fc-9f7d-289386c0ffa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:17:40.197961 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030004d-0a32-42fc-9f7d-289386c0ffa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:17:40.199613 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030004d-0a32-42fc-9f7d-289386c0ffa4 service nova] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:17:40.212115 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1454/2910] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:17:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:17:49.101566 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882888bf-5267-48df-9420-8ec6f1b9ade6 service neutron] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:17:49.113430 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882888bf-5267-48df-9420-8ec6f1b9ade6 service neutron] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:17:49.119250 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fff4e-77bb-4e9e-973d-88d87d344cdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:17:49.119903 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fff4e-77bb-4e9e-973d-88d87d344cdd service placement] REQUEST_METHOD: `GET` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:17:49.120191 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fff4e-77bb-4e9e-973d-88d87d344cdd service placement] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:17:49.120507 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fff4e-77bb-4e9e-973d-88d87d344cdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:17:49.122449 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-c74fff4e-77bb-4e9e-973d-88d87d344cdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:17:49.135276 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fff4e-77bb-4e9e-973d-88d87d344cdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:17:49.136906 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fff4e-77bb-4e9e-973d-88d87d344cdd service placement] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:17:49.147257 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1457/2911] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:17:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:17:50.207251 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=6 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:52.706813 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030004d-0a32-42fc-9f7d-289386c0ffa4 service nova] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:17:52.770296 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030004d-0a32-42fc-9f7d-289386c0ffa4 service nova] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:17:52.774988 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:17:52.775848 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:17:52.776131 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:17:52.776440 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] PATH_INFO: `/v3/users/570744ca9e494bf9901c1552f7a29e7c` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:17:52.777624 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=570744ca9e494bf9901c1552f7a29e7c)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:17:52.779638 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:17:52.970095 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '570744ca9e494bf9901c1552f7a29e7c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:52.970692 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '570744ca9e494bf9901c1552f7a29e7c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:52.982753 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '570744ca9e494bf9901c1552f7a29e7c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:52.983396 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '570744ca9e494bf9901c1552f7a29e7c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:52.990748 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '570744ca9e494bf9901c1552f7a29e7c'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:53.079815 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1455/2912] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:17:52 2022] DELETE /identity/v3/users/570744ca9e494bf9901c1552f7a29e7c => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:17:53.584481 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fff4e-77bb-4e9e-973d-88d87d344cdd service placement] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:17:53.596774 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fff4e-77bb-4e9e-973d-88d87d344cdd service placement] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:17:53.663633 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:17:53.664528 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:17:53.664813 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:17:53.665095 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] PATH_INFO: `/v3/projects/3fe59cde61bd452592e3d52217b1329a` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:17:53.666274 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe59cde61bd452592e3d52217b1329a)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:17:53.668267 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:17:53.777701 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe59cde61bd452592e3d52217b1329a', 'request_id': 'req-1b1e3056-d776-4617-b377-29d2ead89a08'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:53.784067 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe59cde61bd452592e3d52217b1329a', 'request_id': 'req-1b1e3056-d776-4617-b377-29d2ead89a08'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:53.790178 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe59cde61bd452592e3d52217b1329a', 'request_id': 'req-1b1e3056-d776-4617-b377-29d2ead89a08'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:53.857052 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] The token cache is being invalidate because project 3fe59cde61bd452592e3d52217b1329a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:17:53.857268 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:53.857889 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:53.859471 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1458/2913] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:17:53 2022] DELETE /identity/v3/projects/3fe59cde61bd452592e3d52217b1329a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:17:53.863434 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:17:53.865046 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:17:53.878880 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e564b9c-22fb-474c-b4df-afd77bda8105 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:17:53.938878 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:17:53.939762 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:17:53.940049 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:17:53.940330 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] PATH_INFO: `/v3/users/09b408b4a93c48d69824adf9122b183e` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:17:53.941688 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b408b4a93c48d69824adf9122b183e)` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:17:53.944193 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:17:53.967750 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:54.135144 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b408b4a93c48d69824adf9122b183e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:54.135722 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b408b4a93c48d69824adf9122b183e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:54.148503 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b408b4a93c48d69824adf9122b183e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:54.149196 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b408b4a93c48d69824adf9122b183e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:54.156227 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.notifications [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b408b4a93c48d69824adf9122b183e'} {{(pid=87074) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:54.226306 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1456/2914] 10.222.0.23 () {58 vars in 1223 bytes} [Mon May 23 16:17:53 2022] DELETE /identity/v3/users/09b408b4a93c48d69824adf9122b183e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:17:54.473771 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] Authenticating user token {{(pid=87073) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:17:54.474947 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.fernet_utils [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87073) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:17:54.488173 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e3056-d776-4617-b377-29d2ead89a08 admin admin] Validating token access rules against request {{(pid=87073) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:17:54.549290 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0840c2e542b841edbc74432280137f72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6e65bbb7bace4fb6a7f71d68e850edbd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87073) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:17:54.550155 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:17:54.550438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] SCRIPT_NAME: `/identity` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:17:54.550713 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] PATH_INFO: `/v3/projects/b665541eff14427895aa4c5faa87d822` {{(pid=87073) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:17:54.551886 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b665541eff14427895aa4c5faa87d822)` {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:17:54.554450 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] RBAC: Authorization granted {{(pid=87073) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:17:54.575037 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:17:54.656808 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b665541eff14427895aa4c5faa87d822', 'request_id': 'req-47182a58-9bdf-4192-a63e-329ecc3c2440'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:54.663358 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b665541eff14427895aa4c5faa87d822', 'request_id': 'req-47182a58-9bdf-4192-a63e-329ecc3c2440'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:54.668872 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b665541eff14427895aa4c5faa87d822', 'request_id': 'req-47182a58-9bdf-4192-a63e-329ecc3c2440'} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:54.732582 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] The token cache is being invalidate because project b665541eff14427895aa4c5faa87d822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87073) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 16:17:54.732798 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:54.733438 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG keystone.notifications [None req-47182a58-9bdf-4192-a63e-329ecc3c2440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87073) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 16:17:54.735003 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: [pid: 87073|app: 0|req: 1459/2915] 10.222.0.23 () {58 vars in 1232 bytes} [Mon May 23 16:17:54 2022] DELETE /identity/v3/projects/b665541eff14427895aa4c5faa87d822 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:18:04.222482 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:18:04.729059 dv-cinder-842119-2-iscsi devstack@keystone.service[87073]: DEBUG dbcounter [-] [87073] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87073) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:19:25.133997 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] Authenticating user token {{(pid=87074) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:19:25.135384 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:19:25.272242 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc5d39-134e-45b7-b722-086852084852 admin admin] Validating token access rules against request {{(pid=87074) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:19:25.330437 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2f25f-a7f9-41d7-a0b5-36f909657763 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312ed2b555424c6a96b3d972ffea5ac0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b8dd1fc12a644f79405a3ac265147b5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87074) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:19:25.331092 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2f25f-a7f9-41d7-a0b5-36f909657763 service placement] REQUEST_METHOD: `GET` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:19:25.331375 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2f25f-a7f9-41d7-a0b5-36f909657763 service placement] SCRIPT_NAME: `/identity` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:19:25.331674 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2f25f-a7f9-41d7-a0b5-36f909657763 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87074) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:19:25.333965 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.fernet_utils [None req-dff2f25f-a7f9-41d7-a0b5-36f909657763 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87074) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:19:25.345061 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2f25f-a7f9-41d7-a0b5-36f909657763 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:19:25.346624 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2f25f-a7f9-41d7-a0b5-36f909657763 service placement] RBAC: Authorization granted {{(pid=87074) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:19:25.357358 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: [pid: 87074|app: 0|req: 1457/2916] 10.222.0.23 () {60 vars in 1365 bytes} [Mon May 23 16:19:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:19:35.353148 dv-cinder-842119-2-iscsi devstack@keystone.service[87074]: DEBUG dbcounter [-] [87074] Writing DB stats keystone:SELECT=89 {{(pid=87074) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}